ONNX Inference#

Torch-free ONNX inference utilities: Export a torch model to ONNX format and then:

run torch-free forecasting with run_onnx_prediction()
or drive a custom / advanced loop with prepare_onnx_inputs()

class darts.utils.onnx.inference.OnnxModelSpec(input_chunk_length, output_chunk_length, output_chunk_shift, uses_past_covariates, uses_future_covariates, uses_static_covariates, likelihood_parameter_names, feature_input_names, input_names, output_names, state_input_names=<factory>, state_output_names=<factory>, state_input_shapes=None, stepwise_state=False)[source]#

Bases: object

Metadata describing an exported ONNX graph and its forecasting windows.

Embedded in the exported .onnx file under darts.onnx_spec. Feature names match module fields (past_target, optional covariates). Recurrent graphs add flattened state_in_* / state_out_* tensors.

Parameters:

input_chunk_length (int) – Target history consumed per inference window.
output_chunk_length (int) – Steps produced per graph call (1 for stepwise RNN cells).
output_chunk_shift (int) – Steps that inference start is shifted into the future.
uses_past_covariates (bool) – Whether the model was fitted with past covariates.
uses_future_covariates (bool) – Whether the model was fitted with future covariates.
uses_static_covariates (bool) – Whether the model was fitted with static covariates.
likelihood_parameter_names (list[str] | None) – The likelihood parameter names if the model was trained with a likelihood.
feature_input_names (list[str]) – Graph inputs that are features (excludes state_in_*).
input_names (list[str]) – All graph input names, including state.
output_names (list[str]) – All graph output names (prediction plus state_out_*).
state_input_names (list[str]) – Flattened recurrent state inputs, if any.
state_output_names (list[str]) – Flattened recurrent state outputs, if any.
state_input_shapes (list[list[int]] | None) – Shapes of state_input_names (zeros on the first step).
stepwise_state (bool) – If True, the graph is a 1-step cell that must be warmed up over input_chunk_length before forecasting (e.g. RNNModel).

Attributes

state_input_shapes

Methods

from_session(session)

Load the spec embedded in an ONNX Runtime inference session.

feature_input_names: list[str]#

classmethod from_session(session)[source]#

Load the spec embedded in an ONNX Runtime inference session.

Parameters:: session (Any) – An ONNX Runtime InferenceSession for a model exported with to_onnx().
Returns:: The deserialized spec.
Return type:: OnnxModelSpec
Raises:: ValueError – If the model does not contain ONNX_SPEC_METADATA_KEY metadata.

input_chunk_length: int#

input_names: list[str]#

likelihood_parameter_names: list[str] | None#

output_chunk_length: int#

output_chunk_shift: int#

output_names: list[str]#

state_input_names: list[str]#

state_input_shapes: list[list[int]] | None = None#

state_output_names: list[str]#

stepwise_state: bool = False#

uses_future_covariates: bool#

uses_past_covariates: bool#

uses_static_covariates: bool#

darts.utils.onnx.inference.prepare_onnx_inputs(series, spec, past_covariates=None, future_covariates=None, n=None)[source]#

Slice input features for a single ONNX inference window.

Returns a dictionary mapping ONNX input names to numpy arrays with a leading batch dimension. Only includes feature inputs present in spec. Recurrent state_* inputs must be provided separately (or via run_onnx_prediction()).

Parameters:

series (TimeSeries) – The series or sequence of series, representing the history of the target series whose future is to be predicted. If specified, the method returns the forecasts of these series. Otherwise, the method returns the forecast of the (single) training series.
spec (OnnxModelSpec) – Graph / window metadata, typically from OnnxModelSpec.from_session().
past_covariates (TimeSeries | None) – Optionally, the past-observed covariates series needed as inputs for the model. They must match the covariates used for training in terms of dimension.
future_covariates (TimeSeries | None) – Optionally, the future-known covariates series needed as inputs for the model. They must match the covariates used for training in terms of dimension.
n (int | None) – The number of time steps after the end of the target series for which to produce predictions.

Returns:

Feature tensors keyed by ONNX input name, each with a batch dimension.

Return type:

dict[str, np.ndarray]

darts.utils.onnx.inference.run_onnx_prediction(n, session, series, past_covariates=None, future_covariates=None, roll_size=None, verbose=True)[source]#

Run ONNX inference for n steps after series end.

Mirrors TorchForecastingModel.predict() for deterministic models with num_samples=1 and probabilistic models with predict_likelihood_parameters=True.

Autoregressive forecasts are only supported for deterministic models.

Note

Requires ONNX dependencies to be installed:

onnxruntime>=1.24.1 for inference
onnx>=1.0.0 and onnxscript>=0.7.0 for exporting to ONNX

Example for exporting a DLinearModel to ONNX format and torch-free forcasting:

# train torch model and export to ONNX format
from darts.datasets import AirPassengersDataset
from darts.models import DLinearModel

series = AirPassengersDataset().load().astype("f")
model = DLinearModel(input_chunk_length=12, output_chunk_length=1)
model.fit(series, epochs=1)
onnx_filename = "my_model.onnx"
model.to_onnx(onnx_filename)

# run torch-free inference
import onnxruntime as ort
from darts.utils.onnx import run_onnx_prediction

session = ort.InferenceSession(onnx_filename)
forecast = run_onnx_prediction(n=3, session=session, series=series)

Parameters:

n (int) – The number of time steps after the end of the target series for which to produce predictions.
session (Any) – An ONNX Runtime InferenceSession for the exported graph.
series (Union[TimeSeries, Sequence[TimeSeries]]) – The series or sequence of series, representing the history of the target series whose future is to be predicted. If specified, the method returns the forecasts of these series. Otherwise, the method returns the forecast of the (single) training series.
past_covariates (Union[TimeSeries, Sequence[TimeSeries], None]) – Optionally, the past-observed covariates series needed as inputs for the model. They must match the covariates used for training in terms of dimension.
future_covariates (Union[TimeSeries, Sequence[TimeSeries], None]) – Optionally, the future-known covariates series needed as inputs for the model. They must match the covariates used for training in terms of dimension.
roll_size (int | None) – For self-consuming predictions, i.e. n > output_chunk_length, determines how many outputs of the model are fed back into it at every iteration of feeding the predicted target (and optionally future covariates) back into the model. If this parameter is not provided, it will be set output_chunk_length by default. Forced to 1 for stepwise graphs.
verbose (bool) – Whether to display the forecast progress.

Returns:

The forecasted series or sequence of series.

Return type:

TimeSeriesLike

ONNX Inference#

This Page