Metrics

Some metrics to compare time series.

darts.metrics.metrics.ae(actual_series, pred_series, intersect=True, *, q=None, time_reduction=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Absolute Error (AE).

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed per component/column and time step \(t\) as:

\[|y_t - \hat{y}_t|\]

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• time_reduction (Optional[Callable[…, ndarray]]) – Optionally, a function to aggregate the metrics over the time axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (c,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the time axis. If None, will return a metric per time step.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • a sequence (list) of uni/multivariate series with series_reduction, component_reduction and time_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n time steps, n components) without time and component reductions. For:

  • single multivariate series and at least component_reduction=None.

  • single uni/multivariate series and at least time_reduction=None.

  • a sequence of uni/multivariate series including series_reduction and at least one of component_reduction=None or time_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.ape(actual_series, pred_series, intersect=True, *, q=None, time_reduction=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Absolute Percentage Error (APE).

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed as a percentage value per component/column and time step \(t\) with:

\[100 \cdot \left| \frac{y_t - \hat{y}_t}{y_t} \right|\]

Note that it will raise a ValueError if \(y_t = 0\) for some \(t\). Consider using the Absolute Scaled Error (ase()) in these cases.

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• time_reduction (Optional[Callable[…, ndarray]]) – Optionally, a function to aggregate the metrics over the time axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (c,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the time axis. If None, will return a metric per time step.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Raises

ValueError – If actual_series contains some zeros.

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • a sequence (list) of uni/multivariate series with series_reduction, component_reduction and time_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n time steps, n components) without time and component reductions. For:

  • single multivariate series and at least component_reduction=None.

  • single uni/multivariate series and at least time_reduction=None.

  • a sequence of uni/multivariate series including series_reduction and at least one of component_reduction=None or time_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.arre(actual_series, pred_series, intersect=True, *, q=None, time_reduction=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Absolute Ranged Relative Error (ARRE).

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed as a percentage value per component/column and time step \(t\) with:

\[100 \cdot \left| \frac{y_t - \hat{y}_t} {\max_t{y_t} - \min_t{y_t}} \right|\]

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• time_reduction (Optional[Callable[…, ndarray]]) – Optionally, a function to aggregate the metrics over the time axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (c,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the time axis. If None, will return a metric per time step.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Raises

ValueError – If \(\max_t{y_t} = \min_t{y_t}\).

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • a sequence (list) of uni/multivariate series with series_reduction, component_reduction and time_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n time steps, n components) without time and component reductions. For:

  • single multivariate series and at least component_reduction=None.

  • single uni/multivariate series and at least time_reduction=None.

  • a sequence of uni/multivariate series including series_reduction and at least one of component_reduction=None or time_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.ase(actual_series, pred_series, insample, m=1, intersect=True, *, q=None, time_reduction=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Absolute Scaled Error (ASE) (see [1] for more information on scaled forecasting errors).

It is the Absolute Error (AE) scaled by the Mean AE (MAE) of the naive m-seasonal forecast.

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed per component/column and time step \(t\) as:

\[\frac{AE(y_{t_p+1:t_p+T}, \hat{y}_{t_p+1:t_p+T})}{E_m},\]

where \(t_p\) is the prediction time (one step before the first forecasted point), \(AE\) is the Absolute Error (ae()), and \(E_m\) is the Mean AE (MAE) of the naive m-seasonal forecast on the insample series \(y_{0:t_p}\) (the true series ending at \(t_p\)):

\[E_m = MAE(y_{m:t_p}, y_{0:t_p - m}).\]

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• insample (Union[TimeSeries, Sequence[TimeSeries]]) – The training series used to forecast pred_series . This series serves to compute the scale of the error obtained by a naive forecaster on the training data.

• m (int) – The seasonality to use for differencing to compute the error scale \(E_m\) (as described in the metric description). \(m=1\) corresponds to a non-seasonal \(E_m\) (e.g. naive repetition of the last observed value), whereas \(m>1\) corresponds to a seasonal \(E_m\).

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• time_reduction (Optional[Callable[…, ndarray]]) – Optionally, a function to aggregate the metrics over the time axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (c,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the time axis. If None, will return a metric per time step.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Raises

ValueError – If the insample series is periodic ( \(y_t = y_{t-m}\) ) or any series in insample does not end one time step before the start of the corresponding forecast in pred_series.

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • a sequence (list) of uni/multivariate series with series_reduction, component_reduction and time_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n time steps, n components) without time and component reductions. For:

  • single multivariate series and at least component_reduction=None.

  • single uni/multivariate series and at least time_reduction=None.

  • a sequence of uni/multivariate series including series_reduction and at least one of component_reduction=None or time_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

References

1

https://www.pmorgan.com.au/tutorials/mae%2C-mape%2C-mase-and-the-scaled-rmse/

darts.metrics.metrics.coefficient_of_variation(actual_series, pred_series, intersect=True, *, q=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Coefficient of Variation (percentage).

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed per component/column as a percentage value with:

\[100 \cdot \text{RMSE}(y_t, \hat{y}_t) / \bar{y},\]

where \(RMSE\) is the Root Mean Squared Error (rmse()), and \(\bar{y}\) is the average of \(y\) over all time steps.

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • sequence (list) of uni/multivariate series with series_reduction and component_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n components,) without component reduction. For:

  • single multivariate series and at least component_reduction=None.

  • sequence of uni/multivariate series including series_reduction and component_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.dtw_metric(actual_series, pred_series, metric=<function mae>, *, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False, **kwargs)

Applies Dynamic Time Warping to actual_series and pred_series before passing it into the metric. Enables comparison between series of different lengths, phases and time indices.

Defaults to using mae() as a metric.

See dtw() for more supported parameters.

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• metric (Callable[[Union[TimeSeries, Sequence[TimeSeries]], Union[TimeSeries, Sequence[TimeSeries]]], Union[float, List[float], ndarray, List[ndarray]]]) – The selected metric with signature ‘[[TimeSeries, TimeSeries], float]’ to use. Default: mae.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • sequence (list) of uni/multivariate series with series_reduction and component_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n components,) without component reduction. For:

  • single multivariate series and at least component_reduction=None.

  • sequence of uni/multivariate series including series_reduction and component_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.err(actual_series, pred_series, intersect=True, *, q=None, time_reduction=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Error (ERR).

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed per component/column and time step \(t\) as:

\[y_t - \hat{y}_t\]

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• time_reduction (Optional[Callable[…, ndarray]]) – Optionally, a function to aggregate the metrics over the time axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (c,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the time axis. If None, will return a metric per time step.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • a sequence (list) of uni/multivariate series with series_reduction, component_reduction and time_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n time steps, n components) without time and component reductions. For:

  • single multivariate series and at least component_reduction=None.

  • single uni/multivariate series and at least time_reduction=None.

  • a sequence of uni/multivariate series including series_reduction and at least one of component_reduction=None or time_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.interval_support(func)

This decorator adds support for quantile interval metrics with sanity checks, processing, and extraction of quantiles from the intervals.

Return type

Callable[…, Union[float, List[float], ndarray, List[ndarray]]]

darts.metrics.metrics.iw(actual_series, pred_series, intersect=True, *, q_interval=None, q=None, time_reduction=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Interval Width (IL).

IL gives the width of predicted quantile intervals.

For the true series \(y\) and predicted stochastic or quantile series \(\hat{y}\) of length \(T\), it is computed per component/column, quantile interval, and time step \(t\) as:

\[\hat{y}_{t,qh} - \hat{y}_{t,ql}\]

where \(\hat{y}_{t,qh}\) are the upper bound quantile values (of all predicted quantiles or samples) at time \(t\), and \(\hat{y}_{t,ql}\) are the lower bound quantile values.

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q_interval (Union[Tuple[float, float], Sequence[Tuple[float, float]], None]) – The quantile interval(s) to compute the metric on. Must be a tuple (single interval) or sequence tuples (multiple intervals) with elements (low quantile, high quantile).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Quantiles q not supported by this metric; use q_interval instead.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• time_reduction (Optional[Callable[…, ndarray]]) – Optionally, a function to aggregate the metrics over the time axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (c,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the time axis. If None, will return a metric per time step.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • a sequence (list) of uni/multivariate series with series_reduction, component_reduction and time_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n time steps, n components) without time and component reductions. For:

  • single multivariate series and at least component_reduction=None.

  • single uni/multivariate series and at least time_reduction=None.

  • a sequence of uni/multivariate series including series_reduction and at least one of component_reduction=None or time_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.mae(actual_series, pred_series, intersect=True, *, q=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Mean Absolute Error (MAE).

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed per component/column as:

\[\frac{1}{T}\sum_{t=1}^T{|y_t - \hat{y}_t|}\]

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • sequence (list) of uni/multivariate series with series_reduction and component_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n components,) without component reduction. For:

  • single multivariate series and at least component_reduction=None.

  • sequence of uni/multivariate series including series_reduction and component_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.mape(actual_series, pred_series, intersect=True, *, q=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Mean Absolute Percentage Error (MAPE).

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed as a percentage value per component/column with:

\[100 \cdot \frac{1}{T} \sum_{t=1}^{T}{\left| \frac{y_t - \hat{y}_t}{y_t} \right|}\]

Note that it will raise a ValueError if \(y_t = 0\) for some \(t\). Consider using the Mean Absolute Scaled Error (mase()) in these cases.

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Raises

ValueError – If actual_series contains some zeros.

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • sequence (list) of uni/multivariate series with series_reduction and component_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n components,) without component reduction. For:

  • single multivariate series and at least component_reduction=None.

  • sequence of uni/multivariate series including series_reduction and component_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.marre(actual_series, pred_series, intersect=True, *, q=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Mean Absolute Ranged Relative Error (MARRE).

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed as a percentage value per component/column with:

\[100 \cdot \frac{1}{T} \sum_{t=1}^{T} {\left| \frac{y_t - \hat{y}_t} {\max_t{y_t} - \min_t{y_t}} \right|}\]

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Raises

• ValueError – If \(\max_t{y_t} = \min_t{y_t}\).

• float –

  A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • sequence (list) of uni/multivariate series with series_reduction and component_reduction.

• np.ndarray –

  A numpy array of metric scores. The array has shape (n components,) without component reduction. For:

  • single multivariate series and at least component_reduction=None.

  • sequence of uni/multivariate series including series_reduction and component_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

Return type

Union[float, List[float], ndarray, List[ndarray]]

darts.metrics.metrics.mase(actual_series, pred_series, insample, m=1, intersect=True, *, q=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Mean Absolute Scaled Error (MASE) (see [1] for more information on scaled forecasting errors).

It is the Mean Absolute Error (MAE) scaled by the MAE of the naive m-seasonal forecast.

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed per component/column as:

\[\frac{MAE(y_{t_p+1:t_p+T}, \hat{y}_{t_p+1:t_p+T})}{E_m},\]

where \(t_p\) is the prediction time (one step before the first forecasted point), \(MAE\) is the Mean Absolute Error (mae()), and \(E_m\) is the MAE of the naive m-seasonal forecast on the insample series \(y_{0:t_p}\) (the true series ending at \(t_p\)):

\[E_m = MAE(y_{m:t_p}, y_{0:t_p - m}).\]

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• insample (Union[TimeSeries, Sequence[TimeSeries]]) – The training series used to forecast pred_series . This series serves to compute the scale of the error obtained by a naive forecaster on the training data.

• m (int) – The seasonality to use for differencing to compute the error scale \(E_m\) (as described in the metric description). \(m=1\) corresponds to a non-seasonal \(E_m\) (e.g. naive repetition of the last observed value), whereas \(m>1\) corresponds to a seasonal \(E_m\).

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Raises

ValueError – If the insample series is periodic ( \(y_t = y_{t-m}\) ) or any series in insample does not end one time step before the start of the corresponding forecast in pred_series.

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • sequence (list) of uni/multivariate series with series_reduction and component_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n components,) without component reduction. For:

  • single multivariate series and at least component_reduction=None.

  • sequence of uni/multivariate series including series_reduction and component_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

References

1

https://www.pmorgan.com.au/tutorials/mae%2C-mape%2C-mase-and-the-scaled-rmse/

darts.metrics.metrics.merr(actual_series, pred_series, intersect=True, *, q=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Mean Error (MERR).

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed per component/column as:

\[\frac{1}{T}\sum_{t=1}^T{(y_t - \hat{y}_t)}\]

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • sequence (list) of uni/multivariate series with series_reduction and component_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n components,) without component reduction. For:

  • single multivariate series and at least component_reduction=None.

  • sequence of uni/multivariate series including series_reduction and component_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.miw(actual_series, pred_series, intersect=True, *, q_interval=None, q=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Mean Interval Width (IL).

IL gives the width of predicted quantile intervals aggregated over time.

For the true series \(y\) and predicted stochastic or quantile series \(\hat{y}\) of length \(T\), it is computed per component/column, quantile interval, and time step \(t\) as:

\[\frac{1}{T}\sum_{t=1}^T{\hat{y}_{t,qh} - \hat{y}_{t,ql}}\]

where \(\hat{y}_{t,qh}\) are the upper bound quantile values (of all predicted quantiles or samples) at time \(t\), and \(\hat{y}_{t,ql}\) are the lower bound quantile values.

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q_interval (Union[Tuple[float, float], Sequence[Tuple[float, float]], None]) – The quantile interval(s) to compute the metric on. Must be a tuple (single interval) or sequence tuples (multiple intervals) with elements (low quantile, high quantile).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Quantiles q not supported by this metric; use q_interval instead.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • sequence (list) of uni/multivariate series with series_reduction and component_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n components,) without component reduction. For:

  • single multivariate series and at least component_reduction=None.

  • sequence of uni/multivariate series including series_reduction and component_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.mql(actual_series, pred_series, intersect=True, *, q=0.5, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Mean Quantile Loss (MQL).

Also known as Pinball Loss. QL is a metric that quantifies the accuracy of a specific quantile \(q\) from the predicted deterministic quantiles or value distribution of a stochastic/probabilistic pred_series containing N samples.

MQL first computes the quantile of all sample values and the loss per time step, and then takes the mean over the time axis.

For the true series \(y\) and predicted stochastic/probabilistic series (containing N samples) \(\hat{y}\) of of shape \(T \times N\), it is computed per column/component as:

\[2 \frac{1}{T}\sum_{t=1}^T{\max((q - 1) (y_t - \hat{y}_{t,q}), q (y_t - \hat{y}_{t,q}))},\]

where \(\hat{y}_{t,q}\) is quantile value \(q\) (of all predicted quantiles or samples) at time \(t\). The factor 2 makes the loss more interpretable, as for q=0.5 the loss is identical to the Mean Absolute Error (mae()).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index]]) – The quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • sequence (list) of uni/multivariate series with series_reduction and component_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n components,) without component reduction. For:

  • single multivariate series and at least component_reduction=None.

  • sequence of uni/multivariate series including series_reduction and component_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.mse(actual_series, pred_series, intersect=True, *, q=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Mean Squared Error (MSE).

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed per component/column as:

\[\frac{1}{T}\sum_{t=1}^T{(y_t - \hat{y}_t)^2}.\]

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • sequence (list) of uni/multivariate series with series_reduction and component_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n components,) without component reduction. For:

  • single multivariate series and at least component_reduction=None.

  • sequence of uni/multivariate series including series_reduction and component_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.msse(actual_series, pred_series, insample, m=1, intersect=True, *, q=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Mean Squared Scaled Error (MSSE) (see [1] for more information on scaled forecasting errors).

It is the Mean Squared Error (MSE) scaled by the MSE of the naive m-seasonal forecast.

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed per component/column as:

\[\frac{MSE(y_{t_p+1:t_p+T}, \hat{y}_{t_p+1:t_p+T})}{E_m},\]

where \(t_p\) is the prediction time (one step before the first forecasted point), \(MSE\) is the Mean Squared Error (mse()), and \(E_m\) is the MSE of the naive m-seasonal forecast on the insample series \(y_{0:t_p}\) (the true series ending at \(t_p\)):

\[E_m = MSE(y_{m:t_p}, y_{0:t_p - m}).\]

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• insample (Union[TimeSeries, Sequence[TimeSeries]]) – The training series used to forecast pred_series . This series serves to compute the scale of the error obtained by a naive forecaster on the training data.

• m (int) – The seasonality to use for differencing to compute the error scale \(E_m\) (as described in the metric description). \(m=1\) corresponds to a non-seasonal \(E_m\) (e.g. naive repetition of the last observed value), whereas \(m>1\) corresponds to a seasonal \(E_m\).

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Raises

ValueError – If the insample series is periodic ( \(y_t = y_{t-m}\) ) or any series in insample does not end one time step before the start of the corresponding forecast in pred_series.

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • sequence (list) of uni/multivariate series with series_reduction and component_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n components,) without component reduction. For:

  • single multivariate series and at least component_reduction=None.

  • sequence of uni/multivariate series including series_reduction and component_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

References

1

https://www.pmorgan.com.au/tutorials/mae%2C-mape%2C-mase-and-the-scaled-rmse/

darts.metrics.metrics.multi_ts_support(func)

This decorator further adapts the metrics that took as input two (or three for scaled metrics with insample) univariate/multivariate TimeSeries instances, adding support for equally-sized sequences of TimeSeries instances. The decorator computes the pairwise metric for TimeSeries with the same indices, and returns a float value that is computed as a function of all the pairwise metrics using a series_reduction subroutine passed as argument to the metric function.

If a ‘Sequence[TimeSeries]’ is passed as input, this decorator provides also parallelisation of the metric evaluation regarding different TimeSeries (if the n_jobs parameter is not set 1).

Return type

Callable[…, Union[float, List[float], ndarray, List[ndarray]]]

darts.metrics.metrics.multivariate_support(func)

This decorator transforms a metric function that takes as input two univariate TimeSeries instances into a function that takes two equally-sized multivariate TimeSeries instances, computes the pairwise univariate metrics for components with the same indices, and returns a float value that is computed as a function of all the univariate metrics using a component_reduction subroutine passed as argument to the metric function.

Return type

Callable[…, Union[float, List[float], ndarray, List[ndarray]]]

darts.metrics.metrics.ope(actual_series, pred_series, intersect=True, *, q=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Overall Percentage Error (OPE).

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed as a percentage value per component/column with:

\[100 \cdot \left| \frac{\sum_{t=1}^{T}{y_t} - \sum_{t=1}^{T}{\hat{y}_t}}{\sum_{t=1}^{T}{y_t}} \right|.\]

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Raises

ValueError – If \(\sum_{t=1}^{T}{y_t} = 0\).

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • sequence (list) of uni/multivariate series with series_reduction and component_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n components,) without component reduction. For:

  • single multivariate series and at least component_reduction=None.

  • sequence of uni/multivariate series including series_reduction and component_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.ql(actual_series, pred_series, intersect=True, *, q=0.5, time_reduction=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Quantile Loss (QL).

Also known as Pinball Loss. QL is a metric that quantifies the accuracy of a specific quantile \(q\) from the predicted deterministic quantiles or value distribution of a stochastic/probabilistic pred_series containing N samples.

QL computes the quantile of all sample values and the loss per time step.

For the true series \(y\) and predicted stochastic/probabilistic series (containing N samples) \(\hat{y}\) of of shape \(T \times N\), it is computed per column/component and time step \(t\) as:

\[2 \max((q - 1) (y_t - \hat{y}_{t,q}), q (y_t - \hat{y}_{t,q})),\]

where \(\hat{y}_{t,q}\) is quantile value \(q\) (of all predicted quantiles or samples) at time \(t\). The factor 2 makes the loss more interpretable, as for q=0.5 the loss is identical to the Absolute Error (ae()).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index]]) – The quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• time_reduction (Optional[Callable[…, ndarray]]) – Optionally, a function to aggregate the metrics over the time axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (c,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the time axis. If None, will return a metric per time step.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • a sequence (list) of uni/multivariate series with series_reduction, component_reduction and time_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n time steps, n components) without time and component reductions. For:

  • single multivariate series and at least component_reduction=None.

  • single uni/multivariate series and at least time_reduction=None.

  • a sequence of uni/multivariate series including series_reduction and at least one of component_reduction=None or time_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.qr(actual_series, pred_series, intersect=True, *, q=0.5, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Quantile Risk (QR)

QR is a metric that quantifies the accuracy of a specific quantile \(q\) from the predicted value distribution of a stochastic/probabilistic pred_series containing N samples.

The main difference to the Quantile Loss (QL) is that QR computes the quantile and loss on the aggregate of all sample values summed up along the time axis (QL computes the quantile and loss per time step).

For the true series \(y\) and predicted stochastic/probabilistic series (containing N samples) \(\hat{y}\) of of shape \(T \times N\), it is computed per column/component as:

\[2 \frac{QL(Z, \hat{Z}_q)}{Z},\]

where \(QL\) is the Quantile Loss (ql()), \(Z = \sum_{t=1}^{T} y_t\) is the sum of all target/actual values, \(\hat{Z} = \sum_{t=1}^{T} \hat{y}_t\) is the sum of all predicted samples along the time axis, and \(\hat{Z}_q\) is the quantile \(q\) of that sum.

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index]]) – The quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • sequence (list) of uni/multivariate series with series_reduction and component_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n components,) without component reduction. For:

  • single multivariate series and at least component_reduction=None.

  • sequence of uni/multivariate series including series_reduction and component_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.r2_score(actual_series, pred_series, intersect=True, *, q=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Coefficient of Determination \(R^2\) (see [1] for more details).

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed per component/column as:

\[1 - \frac{\sum_{t=1}^T{(y_t - \hat{y}_t)^2}}{\sum_{t=1}^T{(y_t - \bar{y})^2}},\]

where \(\bar{y}\) is the mean of \(y\) over all time steps.

This metric is not symmetric.

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • sequence (list) of uni/multivariate series with series_reduction and component_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n components,) without component reduction. For:

  • single multivariate series and at least component_reduction=None.

  • sequence of uni/multivariate series including series_reduction and component_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

References

1

https://en.wikipedia.org/wiki/Coefficient_of_determination

darts.metrics.metrics.rmse(actual_series, pred_series, intersect=True, *, q=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Root Mean Squared Error (RMSE).

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed per component/column as:

\[\sqrt{\frac{1}{T}\sum_{t=1}^T{(y_t - \hat{y}_t)^2}}\]

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • sequence (list) of uni/multivariate series with series_reduction and component_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n components,) without component reduction. For:

  • single multivariate series and at least component_reduction=None.

  • sequence of uni/multivariate series including series_reduction and component_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.rmsle(actual_series, pred_series, intersect=True, *, q=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Root Mean Squared Log Error (RMSLE).

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed per component/column as:

\[\sqrt{\frac{1}{T}\sum_{t=1}^T{\left(\log{(y_t + 1)} - \log{(\hat{y}_t + 1)}\right)^2}}\]

using the natural logarithm.

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • sequence (list) of uni/multivariate series with series_reduction and component_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n components,) without component reduction. For:

  • single multivariate series and at least component_reduction=None.

  • sequence of uni/multivariate series including series_reduction and component_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.rmsse(actual_series, pred_series, insample, m=1, intersect=True, *, q=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Root Mean Squared Scaled Error (RMSSE) (see [1] for more information on scaled forecasting errors).

It is the Root Mean Squared Error (RMSE) scaled by the RMSE of the naive m-seasonal forecast.

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed per component/column as:

\[\frac{RMSE(y_{t_p+1:t_p+T}, \hat{y}_{t_p+1:t_p+T})}{E_m},\]

where \(t_p\) is the prediction time (one step before the first forecasted point), \(RMSE\) is the Root Mean Squared Error (rmse()), and \(E_m\) is the RMSE of the naive m-seasonal forecast on the insample series \(y_{0:t_p}\) (the true series ending at \(t_p\)):

\[E_m = RMSE(y_{m:t_p}, y_{0:t_p - m}).\]

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• insample (Union[TimeSeries, Sequence[TimeSeries]]) – The training series used to forecast pred_series . This series serves to compute the scale of the error obtained by a naive forecaster on the training data.

• m (int) – The seasonality to use for differencing to compute the error scale \(E_m\) (as described in the metric description). \(m=1\) corresponds to a non-seasonal \(E_m\) (e.g. naive repetition of the last observed value), whereas \(m>1\) corresponds to a seasonal \(E_m\).

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Raises

ValueError – If the insample series is periodic ( \(y_t = y_{t-m}\) ) or any series in insample does not end one time step before the start of the corresponding forecast in pred_series.

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • sequence (list) of uni/multivariate series with series_reduction and component_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n components,) without component reduction. For:

  • single multivariate series and at least component_reduction=None.

  • sequence of uni/multivariate series including series_reduction and component_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

References

1

https://www.pmorgan.com.au/tutorials/mae%2C-mape%2C-mase-and-the-scaled-rmse/

darts.metrics.metrics.sape(actual_series, pred_series, intersect=True, *, q=None, time_reduction=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

symmetric Absolute Percentage Error (sAPE).

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed as a percentage value per component/column and time step \(t\) with:

\[200 \cdot \frac{\left| y_t - \hat{y}_t \right|}{\left| y_t \right| + \left| \hat{y}_t \right|}\]

Note that it will raise a ValueError if \(\left| y_t \right| + \left| \hat{y}_t \right| = 0\) for some \(t\). Consider using the Absolute Scaled Error (ase()) in these cases.

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• time_reduction (Optional[Callable[…, ndarray]]) – Optionally, a function to aggregate the metrics over the time axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (c,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the time axis. If None, will return a metric per time step.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Raises

ValueError – If actual_series and pred_series contain some zeros at the same time index.

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • a sequence (list) of uni/multivariate series with series_reduction, component_reduction and time_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n time steps, n components) without time and component reductions. For:

  • single multivariate series and at least component_reduction=None.

  • single uni/multivariate series and at least time_reduction=None.

  • a sequence of uni/multivariate series including series_reduction and at least one of component_reduction=None or time_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.se(actual_series, pred_series, intersect=True, *, q=None, time_reduction=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Squared Error (SE).

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed per component/column and time step \(t\) as:

\[(y_t - \hat{y}_t)^2.\]

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• time_reduction (Optional[Callable[…, ndarray]]) – Optionally, a function to aggregate the metrics over the time axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (c,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the time axis. If None, will return a metric per time step.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • a sequence (list) of uni/multivariate series with series_reduction, component_reduction and time_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n time steps, n components) without time and component reductions. For:

  • single multivariate series and at least component_reduction=None.

  • single uni/multivariate series and at least time_reduction=None.

  • a sequence of uni/multivariate series including series_reduction and at least one of component_reduction=None or time_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.sle(actual_series, pred_series, intersect=True, *, q=None, time_reduction=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Squared Log Error (SLE).

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed per component/column and time step \(t\) as:

\[\left(\log{(y_t + 1)} - \log{(\hat{y} + 1)}\right)^2\]

using the natural logarithm.

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• time_reduction (Optional[Callable[…, ndarray]]) – Optionally, a function to aggregate the metrics over the time axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (c,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the time axis. If None, will return a metric per time step.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • a sequence (list) of uni/multivariate series with series_reduction, component_reduction and time_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n time steps, n components) without time and component reductions. For:

  • single multivariate series and at least component_reduction=None.

  • single uni/multivariate series and at least time_reduction=None.

  • a sequence of uni/multivariate series including series_reduction and at least one of component_reduction=None or time_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.smape(actual_series, pred_series, intersect=True, *, q=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

symmetric Mean Absolute Percentage Error (sMAPE).

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed as a percentage value per component/column with:

\[200 \cdot \frac{1}{T} \sum_{t=1}^{T}{\frac{\left| y_t - \hat{y}_t \right|}{\left| y_t \right| + \left| \hat{y}_t \right|} }\]

Note that it will raise a ValueError if \(\left| y_t \right| + \left| \hat{y}_t \right| = 0\) for some \(t\). Consider using the Mean Absolute Scaled Error (mase()) in these cases.

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Raises

ValueError – If the actual_series and the pred_series contain some zeros at the same time index.

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • sequence (list) of uni/multivariate series with series_reduction and component_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n components,) without component reduction. For:

  • single multivariate series and at least component_reduction=None.

  • sequence of uni/multivariate series including series_reduction and component_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

darts.metrics.metrics.sse(actual_series, pred_series, insample, m=1, intersect=True, *, q=None, time_reduction=None, component_reduction=<function nanmean>, series_reduction=None, n_jobs=1, verbose=False)

Squared Scaled Error (SSE) (see [1] for more information on scaled forecasting errors).

It is the Squared Error (SE) scaled by the Mean SE (MSE) of the naive m-seasonal forecast.

For the true series \(y\) and predicted series \(\hat{y}\) of length \(T\), it is computed per component/column and time step \(t\) as:

\[\frac{SE(y_{t_p+1:t_p+T}, \hat{y}_{t_p+1:t_p+T})}{E_m},\]

where \(t_p\) is the prediction time (one step before the first forecasted point), \(SE\) is the Squared Error (se()), and \(E_m\) is the Mean SE (MSE) of the naive m-seasonal forecast on the insample series \(y_{0:t_p}\) (the true series ending at \(t_p\)):

\[E_m = MSE(y_{m:t_p}, y_{0:t_p - m}).\]

If \(\hat{y}_t\) are stochastic (contains several samples) or quantile predictions, use parameter q to specify on which quantile(s) to compute the metric on. By default, it uses the median 0.5 quantile (over all samples, or, if given, the quantile prediction itself).

Parameters

• actual_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) actual series.

• pred_series (Union[TimeSeries, Sequence[TimeSeries]]) – The (sequence of) predicted series.

• insample (Union[TimeSeries, Sequence[TimeSeries]]) – The training series used to forecast pred_series . This series serves to compute the scale of the error obtained by a naive forecaster on the training data.

• m (int) – The seasonality to use for differencing to compute the error scale \(E_m\) (as described in the metric description). \(m=1\) corresponds to a non-seasonal \(E_m\) (e.g. naive repetition of the last observed value), whereas \(m>1\) corresponds to a seasonal \(E_m\).

• intersect (bool) – For time series that are overlapping in time without having the same time index, setting True will consider the values only over their common time interval (intersection in time).

• q (Union[float, List[float], Tuple[ndarray, Index], None]) – Optionally, the quantile (float [0, 1]) or list of quantiles of interest to compute the metric on.

• time_reduction (Optional[Callable[…, ndarray]]) – Optionally, a function to aggregate the metrics over the time axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (c,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the time axis. If None, will return a metric per time step.

• component_reduction (Optional[Callable[[ndarray], float]]) – Optionally, a function to aggregate the metrics over the component/column axis. It must reduce a np.ndarray of shape (t, c) to a np.ndarray of shape (t,). The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 1 corresponding to the component axis. If None, will return a metric per component.

• series_reduction (Optional[Callable[[ndarray], Union[float, ndarray]]]) – Optionally, a function to aggregate the metrics over multiple series. It must reduce a np.ndarray of shape (s, t, c) to a np.ndarray of shape (t, c) The function takes as input a np.ndarray and a parameter named axis, and returns the reduced array. The axis receives value 0 corresponding to the series axis. For example with np.nanmean, will return the average over all series metrics. If None, will return a metric per component.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input, parallelising operations regarding different TimeSeries. Defaults to 1 (sequential). Setting the parameter to -1 means using all the available processors.

• verbose (bool) – Optionally, whether to print operations progress

Raises

ValueError – If the insample series is periodic ( \(y_t = y_{t-m}\) ) or any series in insample does not end one time step before the start of the corresponding forecast in pred_series.

Return type

Union[float, List[float], ndarray, List[ndarray]]

Returns

• float – A single metric score for:

  • single univariate series.

  • single multivariate series with component_reduction.

  • a sequence (list) of uni/multivariate series with series_reduction, component_reduction and time_reduction.

• np.ndarray – A numpy array of metric scores. The array has shape (n time steps, n components) without time and component reductions. For:

  • single multivariate series and at least component_reduction=None.

  • single uni/multivariate series and at least time_reduction=None.

  • a sequence of uni/multivariate series including series_reduction and at least one of component_reduction=None or time_reduction=None.

• List[float] – Same as for type float but for a sequence of series.

• List[np.ndarray] – Same as for type np.ndarray but for a sequence of series.

References

1

https://www.pmorgan.com.au/tutorials/mae%2C-mape%2C-mase-and-the-scaled-rmse/

previous

Metrics

next

Models