Window Transformer

class darts.dataprocessing.transformers.window_transformer.WindowTransformer(transforms, treat_na=None, forecasting_safe=True, keep_non_transformed=False, include_current=True, keep_names=False, name='WindowTransformer', n_jobs=1, verbose=False)

Bases: BaseDataTransformer

A transformer that applies window transformation to a TimeSeries or a Sequence of TimeSeries. It expects a dictionary or a list of dictionaries specifying the window transformation(s) to be applied. All series in the sequence will be transformed with the same transformations.

Parameters

• transforms (Union[dict, List[dict]]) –

  A dictionary or a list of dictionaries. Each dictionary specifies a different window transform.

  The dictionaries can contain the following keys:

  "function"

  Mandatory. The name of one of the pandas builtin transformation functions, or a callable function that can be applied to the input series. Pandas’ functions can be found in the documentation.

  "mode"

  Optional. The name of the pandas windowing mode on which the "function" is going to be applied. The options are “rolling”, “expanding” and “ewm”. If not provided, Darts defaults to “expanding”. User defined functions can use either “rolling” or “expanding” modes. More information on pandas windowing operations can be found in the documentation.

  "components"

  Optional. A string or list of strings specifying the TimeSeries components on which the transformation should be applied. If not specified, the transformation will be applied on all components.

  "function_name"

  Optional. A string specifying the function name referenced as part of the transformation output name. For example, given a user-provided function transformation on rolling window size of 5 on the component “comp”, the default transformation output name is “rolling_udf_5_comp” whereby “udf” refers to “user defined function”. If specified, the "function_name" will replace the default name “udf”. Similarly, the "function_name" will replace the name of the pandas builtin transformation function name in the output name.

  All other dictionary items provided will be treated as keyword arguments for the windowing mode (i.e., rolling/ewm/expanding) or for the specific function in that mode (i.e., pandas.DataFrame.rolling.mean/std/max/min... or pandas.DataFrame.ewm.mean/std/sum). This allows for more flexibility in configuring the transformation, by providing for example:

  • "window"

    Size of the moving window for the “rolling” mode. If an integer, the fixed number of observations used for each window. If an offset, the time period of each window with data type pandas.Timedelta representing a fixed duration.

  • "min_periods"

    The minimum number of observations in the window required to have a value (otherwise NaN). Darts reuses pandas defaults of 1 for “rolling” and “expanding” modes and of 0 for “ewm” mode.

  • "win_type"

    The type of weigthing to apply to the window elements. If provided, it should be one of scipy.signal.windows.

  • "center"

    True/False to set the observation at the current timestep at the center of the window (when forecasting_safe is True, Darts enforces "center" to False).

  • "closed"

    "right"/"left"/"both"/"neither" to specify whether the right, left or both ends of the window are included in the window, or neither of them. Darts defaults to "both".

  More information on the available functions and their parameters can be found in the Pandas documentation.

  For user-provided functions, extra keyword arguments in the transformation dictionary are passed to the user-defined function. By default, Darts expects user-defined functions to receive numpy arrays as input. This can be modified by adding item "raw": False in the transformation dictionary. It is expected that the function returns a single value for each window. Other possible configurations can be found in the pandas.DataFrame.rolling().apply() documentation and pandas.DataFrame.expanding().apply() documentation.

• treat_na (Union[str, int, float, None]) –

  Specifies how to treat missing values that were added by the window transformations at the beginning of the resulting TimeSeries. By default, Darts will leave NaNs in the resulting TimeSeries. This parameter can be one of the following:

  • "dropna"

    to truncate the TimeSeries and drop time steps containing missing values. If multiple columns contain different numbers of missing values, only the minimum number of rows is dropped. This operation might reduce the length of the resulting TimeSeries.

  • "bfill" or "backfill"

    to specify that NaNs should be filled with the last transformed and valid observation. If the original TimeSeries starts with NaNs, those are kept. When forecasting_safe is True, this option returns an exception to avoid future observation contaminating the past.

  • an integer or float

    in which case NaNs will be filled with this value. All columns will be filled with the same provided value.

• forecasting_safe (Optional[bool]) – If True, Darts enforces that the resulting TimeSeries is safe to be used in forecasting models as target or as feature. The window transformation will not allow future values to be included in the computations at their corresponding current timestep. Default is True. “ewm” and “expanding” modes are forecasting safe by default. “rolling” mode is forecasting safe if "center": False is guaranteed.

• keep_non_transformed (Optional[bool]) – False to return the transformed components only, True to return all original components along the transformed ones. Default is False. If the series has a hierarchy, must be set to False.

• include_current (Optional[bool]) – True to include the current time step in the window, False to exclude it. Default is True.

• keep_names (Optional[bool]) – Whether the transformed components should keep the original component names or. Must be set to False if keep_non_transformed = True or the number of transformation is greater than 1.

• name (str) – A specific name for the transformer.

• n_jobs (int) – The number of jobs to run in parallel. Parallel jobs are created only when a Sequence[TimeSeries] is passed as input to a method, parallelising operations regarding different TimeSeries. Defaults to 1.

• verbose (bool) – Whether to print operations progress.

Attributes

name	Name of the data transformer.

Methods

apply_component_mask(series[, ...])	Extracts components specified by component_mask from series
set_n_jobs(value)	Set the number of processors to be used by the transformer while processing multiple TimeSeries.
set_verbose(value)	Set the verbosity status.
stack_samples(vals)	Creates an array of shape (n_timesteps * n_samples, n_components) from either a TimeSeries or the array_values of a TimeSeries.
transform(series, *args[, component_mask])	Transforms a (sequence of) of series by calling the user-implemeneted ts_transform method.
ts_transform(series, params)	The function that will be applied to each series when transform() is called.
unapply_component_mask(series, vals[, ...])	Adds back components previously removed by component_mask in apply_component_mask method.
unstack_samples(vals[, n_timesteps, ...])	Reshapes the 2D array returned by stack_samples back into an array of shape (n_timesteps, n_components, n_samples); this 'undoes' the reshaping of stack_samples.

static apply_component_mask(series, component_mask=None, return_ts=False)

Extracts components specified by component_mask from series

Parameters

• series (TimeSeries) – input TimeSeries to be fed into transformer.

• component_mask (Optional[ndarray]) – Optionally, np.ndarray boolean mask of shape (n_components, 1) specifying which components to extract from series. The i`th component of `series is kept only if component_mask[i] = True. If not specified, no masking is performed.

• return_ts (bool) – Optionally, specifies that a TimeSeries should be returned, rather than an np.ndarray.

Returns

TimeSeries (if return_ts = True) or np.ndarray (if return_ts = False) with only those components specified by component_mask remaining.

Return type

masked

property name

Name of the data transformer.

set_n_jobs(value)

Set the number of processors to be used by the transformer while processing multiple TimeSeries.

Parameters

value (int) – New n_jobs value. Set to -1 for using all the available cores.

set_verbose(value)

Set the verbosity status.

True for enabling the detailed report about scaler’s operation progress, False for no additional information.

Parameters

value (bool) – New verbosity status

static stack_samples(vals)

Creates an array of shape (n_timesteps * n_samples, n_components) from either a TimeSeries or the array_values of a TimeSeries.

Each column of the returned array corresponds to a component (dimension) of the series and is formed by concatenating all of the samples associated with that component together. More specifically, the i`th column is formed by concatenating `[component_i_sample_1, component_i_sample_2, …, component_i_sample_n].

Stacking is useful when implementing a transformation that applies the exact same change to every timestep in the timeseries. In such cases, the samples of each component can be stacked together into a single column, and the transformation can then be applied to each column, thereby ‘vectorising’ the transformation over all samples of that component; the unstack_samples method can then be used to reshape the output. For transformations that depend on the time_index or the temporal ordering of the observations, stacking should not be employed.

Parameters

vals (Union[ndarray, TimeSeries]) – Timeseries or np.ndarray of shape (n_timesteps, n_components, n_samples) to be ‘stacked’.

Returns

np.ndarray of shape (n_timesteps * n_samples, n_components), where the i`th column is formed by concatenating all of the samples of the `i`th component in `vals.

Return type

stacked

transform(series, *args, component_mask=None, **kwargs)

Transforms a (sequence of) of series by calling the user-implemeneted ts_transform method.

In case a Sequence[TimeSeries] is passed as input data, this function takes care of parallelising the transformation of multiple series in the sequence at the same time. Additionally, if the mask_components attribute was set to True when instantiating BaseDataTransformer, then any provided component_mask`s will be automatically applied to each input `TimeSeries; please refer to ‘Notes’ for further details on component masking.

Any additionally specified *args and **kwargs are automatically passed to ts_transform.

Parameters

• series (Union[TimeSeries, Sequence[TimeSeries]]) – (sequence of) series to be transformed.

• args – Additional positional arguments for each ts_transform() method call

• component_mask (Optional[np.ndarray] = None) – Optionally, a 1-D boolean np.ndarray of length series.n_components that specifies which components of the underlying series the transform should consider. If the mask_components attribute was set to True when instantiating BaseDataTransformer, then the component mask will be automatically applied to each TimeSeries input. Otherwise, component_mask will be provided as an addition keyword argument to ts_transform. See ‘Notes’ for further details.

• kwargs – Additional keyword arguments for each ts_transform() method call

Returns

Transformed data.

Return type

Union[TimeSeries, List[TimeSeries]]

Notes

If the mask_components attribute was set to True when instantiating BaseDataTransformer, then any provided component_mask`s will be automatically applied to each `TimeSeries input to transform; component_mask`s are simply boolean arrays of shape `(series.n_components,) that specify which components of each series should be transformed using ts_transform and which components should not. If component_mask[i] is True, then the i`th component of each `series will be transformed by ts_transform. Conversely, if component_mask[i] is False, the i`th component will be removed from each `series before being passed to ts_transform; after transforming this masked series, the untransformed i`th component will be ‘added back’ to the output. Note that automatic `component_mask`ing can only be performed if the `ts_transform does not change the number of timesteps in each series; if this were to happen, then the transformed and untransformed components are unable to be concatenated back together along the component axis.

If mask_components was set to False when instantiating BaseDataTransformer, then any provided component_masks will be passed as a keyword argument ts_transform; the user can then manually specify how the component_mask should be applied to each series.

static ts_transform(series, params)

The function that will be applied to each series when transform() is called.

This method is not implemented in the base class and must be implemented in the deriving classes.

The function must take as first argument a TimeSeries object and, as a second argument, a dictionary containing the fixed and/or fitted parameters of the transformation; this function should then return a transformed TimeSeries object.

The params dictionary can contain up to two keys:

1. params[‘fixed’] stores the fixed parameters of the transformation (i.e. attributed defined in the __init__ method of the child-most class before super().__init__ is called); params[‘fixed’] is a dictionary itself, whose keys are the names of the fixed parameter attributes. For example, if _my_fixed_param is defined as an attribute in the child-most class, then this fixed parameter value can be accessed through params[‘fixed’][‘_my_fixed_param’]. 2. If the transform inherits from the FittableDataTransformer class, then params[‘fitted’] will store the fitted parameters of the transformation; the fitted parameters are simply the output(s) returned by the ts_fit function, whatever those output(s) may be. See FittableDataTransformer for further details about fitted parameters.

Any positional/keyword argument supplied to the transform method are passed as positional/keyword arguments to ts_transform; hence, ts_transform should also accept *args and/or **kwargs if positional/keyword arguments are passed to transform. Note that if the mask_components attribute of BaseDataTransformer is set to False, then the component_mask provided to transform will be passed as an additional keyword argument to ts_transform.

The BaseDataTransformer class includes some helper methods which may prove useful when implementing a ts_transform function:

1. The apply_component_mask and unapply_component_mask methods, which apply and ‘unapply’ component_mask`s to a `TimeSeries respectively; these methods are automatically called in transform if the mask_component attribute of BaseDataTransformer is set to True, but you may want to manually call them if you set mask_components to False and wish to manually specify how component_mask`s are applied to a `TimeSeries. 2. The stack_samples method, which stacks all the samples in a TimeSeries along the component axis, so that the TimeSeries goes from shape (n_timesteps, n_components, n_samples) to shape (n_timesteps, n_components * n_samples). This stacking is useful if a pointwise transform is being implemented (i.e. transforming the value at time t depends only on the value of the series at that time t). Once transformed, the stacked TimeSeries can be ‘unstacked’ using the unstack_samples method.

Parameters

• series (TimeSeries) – series to be transformed.

• params (Mapping[str, Any]) – Dictionary containing the parameters of the transformation function. Fixed parameters (i.e. attributes defined in the child-most class of the transformation prior to calling super.__init__()) are stored under the ‘fixed’ key. If the transformation inherits from the FittableDataTransformer class, then the fitted parameters of the transformation (i.e. the values returned by ts_fit) are stored under the ‘fitted’ key.

• args – Any poisitional arguments provided in addition to series when

• kwargs – Any additional keyword arguments provided to transform. Note that if the mask_component attribute of BaseDataTransformer is set to False, then component_mask will be passed as a keyword argument.

Notes

This method is designed to be a static method instead of instance method to allow an efficient parallelisation also when the scaler instance is storing a non-negligible amount of data. Using instance methods would imply copying the instance’s data through multiple processes, which can easily introduce a bottleneck and nullify parallelisation benefits.

Return type

TimeSeries

static unapply_component_mask(series, vals, component_mask=None)

Adds back components previously removed by component_mask in apply_component_mask method.

Parameters

• series (Union[TimeSeries, Sequence[TimeSeries]]) – input TimeSeries that was fed into transformer.

• vals (Union[ndarray, Sequence[ndarray], TimeSeries, Sequence[TimeSeries]]) – np.ndarray or TimeSeries to ‘unmask’

• component_mask (Optional[ndarray]) – Optionally, np.ndarray boolean mask of shape (n_components, 1) specifying which components were extracted from series. If given, insert vals back into the columns of the original array. If not specified, nothing is ‘unmasked’.

Returns

TimeSeries (if vals is a TimeSeries) or np.ndarray (if vals is an np.ndarray) with those components previously removed by component_mask now ‘added back’.

Return type

unmasked

static unstack_samples(vals, n_timesteps=None, n_samples=None, series=None)

Reshapes the 2D array returned by stack_samples back into an array of shape (n_timesteps, n_components, n_samples); this ‘undoes’ the reshaping of stack_samples. Either n_components, n_samples, or series must be specified.

Parameters

• vals (ndarray) – np.ndarray of shape (n_timesteps * n_samples, n_components) to be ‘unstacked’.

• n_timesteps (Optional[int]) – Optionally, the number of timesteps in the array originally passed to stack_samples. Does not need to be provided if series is specified.

• n_samples (Optional[int]) – Optionally, the number of samples in the array originally passed to stack_samples. Does not need to be provided if series is specified.

• series (Optional[TimeSeries]) – Optionally, the TimeSeries object used to create vals; n_samples is inferred from this.

Returns

np.ndarray of shape (n_timesteps, n_components, n_samples).

Return type

unstacked

previous

Static Covariates Transformer

next

Pipeline