{ "cells": [ { "cell_type": "markdown", "id": "8a7af915", "metadata": {}, "source": [ "# MLflow Integration\n", "\n", "This notebook walks through a practical MLOps workflow with Darts' native MLflow integration: enable autolog, compare forecasting models, promote the best one to the registry, and run inference from a production alias.\n", "\n", "If you are new to Darts, see the [Quickstart Guide](https://unit8co.github.io/darts/quickstart/00-quickstart.html) first.\n", "\n", "**Prerequisites:** install MLflow as an optional dependency:\n", "\n", "```bash\n", "pip install \"mlflow>=3.0\"\n", "```\n", "\n", "API reference: [darts.utils.mlflow](https://unit8co.github.io/darts/generated_api/darts.utils.mlflow.html)." ] }, { "cell_type": "code", "execution_count": 1, "id": "7b28941c", "metadata": {}, "outputs": [], "source": [ "%load_ext autoreload\n", "%autoreload 2" ] }, { "cell_type": "code", "execution_count": null, "id": "11bca7fd", "metadata": {}, "outputs": [ { "data": { "text/html": [ " \n", " \n", " " ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import os\n", "import tempfile\n", "import warnings\n", "\n", "import mlflow\n", "import plotly\n", "from mlflow import MlflowClient\n", "\n", "import darts.metrics as metrics\n", "from darts import set_option\n", "from darts.datasets import AirPassengersDataset\n", "from darts.models import ExponentialSmoothing, LinearRegressionModel\n", "from darts.utils.mlflow import autolog, load_model\n", "\n", "warnings.filterwarnings(\"ignore\", category=FutureWarning)\n", "set_option(\"plotting.use_darts_style\", True)\n", "plotly.offline.init_notebook_mode()\n", "\n", "PLOTLY_KWARGS = dict(\n", " legend=dict(yanchor=\"top\", y=0.99, xanchor=\"left\", x=0.01),\n", ")" ] }, { "cell_type": "markdown", "id": "4eb6fc63", "metadata": {}, "source": [ "## 1. MLflow setup\n", "\n", "Point MLflow at a tracking backend and create an experiment. We use a temporary directory so this notebook runs self-contained: run metadata goes to a SQLite database, and artifacts (models, JSON files) are stored alongside it. In production, set `tracking_uri` to your team's MLflow server or local database." ] }, { "cell_type": "code", "execution_count": 3, "id": "7aced0df", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2026/09/04 14:03:54 INFO mlflow.store.db.utils: Creating initial MLflow database tables...\n", "2026/09/04 14:03:54 INFO mlflow.store.db.utils: Updating database tables\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Tracking URI: sqlite:////var/folders/4h/l09drklx06g8022q7khgyxyr0000gn/T/tmpjnang74m/mlflow.db\n", "Experiment: darts-mlflow-examples\n", "\n", "To explore runs in the UI:\n", " mlflow ui --backend-store-uri sqlite:////var/folders/4h/l09drklx06g8022q7khgyxyr0000gn/T/tmpjnang74m/mlflow.db\n" ] } ], "source": [ "tmpdir = tempfile.mkdtemp()\n", "mlflow_db = os.path.join(tmpdir, \"mlflow.db\")\n", "artifact_root = os.path.join(tmpdir, \"mlruns\")\n", "\n", "EXPERIMENT_NAME = \"darts-mlflow-examples\"\n", "mlflow.set_tracking_uri(f\"sqlite:///{mlflow_db}\")\n", "mlflow.set_experiment(\n", " experiment_id=mlflow.create_experiment(\n", " EXPERIMENT_NAME,\n", " artifact_location=artifact_root,\n", " )\n", ")\n", "\n", "print(f\"Tracking URI: {mlflow.get_tracking_uri()}\")\n", "print(f\"Experiment: {mlflow.get_experiment_by_name(EXPERIMENT_NAME).name}\")\n", "print(\n", " f\"\\nTo explore runs in the UI:\\n mlflow ui --backend-store-uri sqlite:///{mlflow_db}\"\n", ")" ] }, { "cell_type": "markdown", "id": "72438a10", "metadata": {}, "source": [ "## 2. Load data\n", "\n", "We use the classic AirPassengers dataset. The last 36 months are held-out for evaluation using a rolling backtest." ] }, { "cell_type": "code", "execution_count": 4, "id": "f7e5ffa6", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Training: 108 points | Validation: 36 points\n" ] }, { "data": { "application/vnd.plotly.v1+json": { "data": [ { "hovertemplate": "#Passengers: %{y:.3g}", "legendgroup": "#Passengers", "line": { "color": "#000000" }, "mode": "lines", "name": "#Passengers", "type": "scatter", "x": [ "1949-01-01T00:00:00.000000000", "1949-02-01T00:00:00.000000000", "1949-03-01T00:00:00.000000000", "1949-04-01T00:00:00.000000000", "1949-05-01T00:00:00.000000000", "1949-06-01T00:00:00.000000000", "1949-07-01T00:00:00.000000000", "1949-08-01T00:00:00.000000000", "1949-09-01T00:00:00.000000000", "1949-10-01T00:00:00.000000000", "1949-11-01T00:00:00.000000000", "1949-12-01T00:00:00.000000000", "1950-01-01T00:00:00.000000000", "1950-02-01T00:00:00.000000000", "1950-03-01T00:00:00.000000000", "1950-04-01T00:00:00.000000000", "1950-05-01T00:00:00.000000000", "1950-06-01T00:00:00.000000000", "1950-07-01T00:00:00.000000000", "1950-08-01T00:00:00.000000000", "1950-09-01T00:00:00.000000000", "1950-10-01T00:00:00.000000000", "1950-11-01T00:00:00.000000000", "1950-12-01T00:00:00.000000000", "1951-01-01T00:00:00.000000000", "1951-02-01T00:00:00.000000000", "1951-03-01T00:00:00.000000000", "1951-04-01T00:00:00.000000000", "1951-05-01T00:00:00.000000000", "1951-06-01T00:00:00.000000000", "1951-07-01T00:00:00.000000000", "1951-08-01T00:00:00.000000000", "1951-09-01T00:00:00.000000000", "1951-10-01T00:00:00.000000000", "1951-11-01T00:00:00.000000000", "1951-12-01T00:00:00.000000000", "1952-01-01T00:00:00.000000000", "1952-02-01T00:00:00.000000000", "1952-03-01T00:00:00.000000000", "1952-04-01T00:00:00.000000000", "1952-05-01T00:00:00.000000000", "1952-06-01T00:00:00.000000000", "1952-07-01T00:00:00.000000000", "1952-08-01T00:00:00.000000000", "1952-09-01T00:00:00.000000000", "1952-10-01T00:00:00.000000000", "1952-11-01T00:00:00.000000000", "1952-12-01T00:00:00.000000000", "1953-01-01T00:00:00.000000000", "1953-02-01T00:00:00.000000000", "1953-03-01T00:00:00.000000000", "1953-04-01T00:00:00.000000000", "1953-05-01T00:00:00.000000000", "1953-06-01T00:00:00.000000000", "1953-07-01T00:00:00.000000000", "1953-08-01T00:00:00.000000000", "1953-09-01T00:00:00.000000000", "1953-10-01T00:00:00.000000000", "1953-11-01T00:00:00.000000000", "1953-12-01T00:00:00.000000000", "1954-01-01T00:00:00.000000000", "1954-02-01T00:00:00.000000000", "1954-03-01T00:00:00.000000000", "1954-04-01T00:00:00.000000000", "1954-05-01T00:00:00.000000000", "1954-06-01T00:00:00.000000000", "1954-07-01T00:00:00.000000000", "1954-08-01T00:00:00.000000000", "1954-09-01T00:00:00.000000000", "1954-10-01T00:00:00.000000000", "1954-11-01T00:00:00.000000000", "1954-12-01T00:00:00.000000000", "1955-01-01T00:00:00.000000000", "1955-02-01T00:00:00.000000000", "1955-03-01T00:00:00.000000000", "1955-04-01T00:00:00.000000000", "1955-05-01T00:00:00.000000000", "1955-06-01T00:00:00.000000000", "1955-07-01T00:00:00.000000000", "1955-08-01T00:00:00.000000000", "1955-09-01T00:00:00.000000000", "1955-10-01T00:00:00.000000000", "1955-11-01T00:00:00.000000000", "1955-12-01T00:00:00.000000000", "1956-01-01T00:00:00.000000000", "1956-02-01T00:00:00.000000000", "1956-03-01T00:00:00.000000000", "1956-04-01T00:00:00.000000000", "1956-05-01T00:00:00.000000000", "1956-06-01T00:00:00.000000000", "1956-07-01T00:00:00.000000000", "1956-08-01T00:00:00.000000000", "1956-09-01T00:00:00.000000000", "1956-10-01T00:00:00.000000000", "1956-11-01T00:00:00.000000000", "1956-12-01T00:00:00.000000000", "1957-01-01T00:00:00.000000000", "1957-02-01T00:00:00.000000000", "1957-03-01T00:00:00.000000000", "1957-04-01T00:00:00.000000000", "1957-05-01T00:00:00.000000000", "1957-06-01T00:00:00.000000000", "1957-07-01T00:00:00.000000000", "1957-08-01T00:00:00.000000000", "1957-09-01T00:00:00.000000000", "1957-10-01T00:00:00.000000000", "1957-11-01T00:00:00.000000000", "1957-12-01T00:00:00.000000000", "1958-01-01T00:00:00.000000000", "1958-02-01T00:00:00.000000000", "1958-03-01T00:00:00.000000000", "1958-04-01T00:00:00.000000000", "1958-05-01T00:00:00.000000000", "1958-06-01T00:00:00.000000000", "1958-07-01T00:00:00.000000000", "1958-08-01T00:00:00.000000000", "1958-09-01T00:00:00.000000000", "1958-10-01T00:00:00.000000000", "1958-11-01T00:00:00.000000000", "1958-12-01T00:00:00.000000000", "1959-01-01T00:00:00.000000000", "1959-02-01T00:00:00.000000000", "1959-03-01T00:00:00.000000000", "1959-04-01T00:00:00.000000000", "1959-05-01T00:00:00.000000000", "1959-06-01T00:00:00.000000000", "1959-07-01T00:00:00.000000000", "1959-08-01T00:00:00.000000000", "1959-09-01T00:00:00.000000000", "1959-10-01T00:00:00.000000000", "1959-11-01T00:00:00.000000000", "1959-12-01T00:00:00.000000000", "1960-01-01T00:00:00.000000000", "1960-02-01T00:00:00.000000000", "1960-03-01T00:00:00.000000000", "1960-04-01T00:00:00.000000000", "1960-05-01T00:00:00.000000000", "1960-06-01T00:00:00.000000000", "1960-07-01T00:00:00.000000000", "1960-08-01T00:00:00.000000000", "1960-09-01T00:00:00.000000000", "1960-10-01T00:00:00.000000000", "1960-11-01T00:00:00.000000000", "1960-12-01T00:00:00.000000000" ], "y": { "bdata": "AAAAAAAAXEAAAAAAAIBdQAAAAAAAgGBAAAAAAAAgYEAAAAAAAEBeQAAAAAAA4GBAAAAAAACAYkAAAAAAAIBiQAAAAAAAAGFAAAAAAADAXUAAAAAAAABaQAAAAAAAgF1AAAAAAADAXEAAAAAAAIBfQAAAAAAAoGFAAAAAAADgYEAAAAAAAEBfQAAAAAAAoGJAAAAAAABAZUAAAAAAAEBlQAAAAAAAwGNAAAAAAACgYEAAAAAAAIBcQAAAAAAAgGFAAAAAAAAgYkAAAAAAAMBiQAAAAAAAQGZAAAAAAABgZEAAAAAAAIBlQAAAAAAAQGZAAAAAAADgaEAAAAAAAOBoQAAAAAAAAGdAAAAAAABAZEAAAAAAAEBiQAAAAAAAwGRAAAAAAABgZUAAAAAAAIBmQAAAAAAAIGhAAAAAAACgZkAAAAAAAOBmQAAAAAAAQGtAAAAAAADAbEAAAAAAAEBuQAAAAAAAIGpAAAAAAADgZ0AAAAAAAIBlQAAAAAAAQGhAAAAAAACAaEAAAAAAAIBoQAAAAAAAgG1AAAAAAABgbUAAAAAAAKBsQAAAAAAAYG5AAAAAAACAcEAAAAAAAABxQAAAAAAAoG1AAAAAAABgakAAAAAAAIBmQAAAAAAAIGlAAAAAAACAaUAAAAAAAIBnQAAAAAAAYG1AAAAAAABgbEAAAAAAAEBtQAAAAAAAgHBAAAAAAADgckAAAAAAAFByQAAAAAAAMHBAAAAAAACgbEAAAAAAAGBpQAAAAAAAoGxAAAAAAABAbkAAAAAAACBtQAAAAAAAsHBAAAAAAADQcEAAAAAAAOBwQAAAAAAAsHNAAAAAAADAdkAAAAAAALB1QAAAAAAAgHNAAAAAAAAgcUAAAAAAAKBtQAAAAAAAYHFAAAAAAADAcUAAAAAAAFBxQAAAAAAA0HNAAAAAAACQc0AAAAAAAOBzQAAAAAAAYHdAAAAAAADQeUAAAAAAAFB5QAAAAAAAMHZAAAAAAAAgc0AAAAAAAPBwQAAAAAAAIHNAAAAAAACwc0AAAAAAANByQAAAAAAAQHZAAAAAAADAdUAAAAAAADB2QAAAAAAAYHpAAAAAAAAQfUAAAAAAADB9QAAAAAAAQHlAAAAAAACwdUAAAAAAABBzQAAAAAAAAHVAAAAAAABAdUAAAAAAAOBzQAAAAAAAoHZAAAAAAADAdUAAAAAAALB2QAAAAAAAMHtAAAAAAACwfkAAAAAAAJB/QAAAAAAAQHlAAAAAAABwdkAAAAAAAGBzQAAAAAAAEHVAAAAAAACAdkAAAAAAAGB1QAAAAAAAYHlAAAAAAADAeEAAAAAAAEB6QAAAAAAAgH1AAAAAAAAggUAAAAAAAHiBQAAAAAAA8HxAAAAAAABweUAAAAAAAKB2QAAAAAAAUHlAAAAAAAAQekAAAAAAAHB4QAAAAAAAMHpAAAAAAADQfEAAAAAAAIB9QAAAAAAAuIBAAAAAAABwg0AAAAAAAPCCQAAAAAAAwH9AAAAAAADQfEAAAAAAAGB4QAAAAAAAAHtA", "dtype": "f8" } } ], "layout": { "annotations": [ { "showarrow": false, "text": "Train / val split ", "x": "1957-12-01T00:00:00", "xanchor": "right", "xref": "x", "y": 1, "yanchor": "top", "yref": "y domain" } ], "hovermode": "x unified", "legend": { "x": 0.01, "xanchor": "left", "y": 0.99, "yanchor": "top" }, "shapes": [ { "line": { "color": "blue", "dash": "dash" }, "type": "line", "x0": "1957-12-01T00:00:00", "x1": "1957-12-01T00:00:00", "xref": "x", "y0": 0, "y1": 1, "yref": "y domain" } ], "template": { "data": { "scatter": [ { "line": { "width": 3 }, "type": "scatter" } ] }, "layout": { "colorway": [ "#000000", "#003dfd", "#b512b8", "#11a9ba", "#0d780f", "#f77f07", "#ba0f0f" ], "font": { "color": "black", "family": "Arial, sans-serif", "size": 14 }, "legend": { "bgcolor": "rgba(255, 255, 255, 0.8)", "borderwidth": 0, "font": { "size": 14 }, "x": 1, "xanchor": "right", "y": 1, "yanchor": "top" }, "margin": { "b": 50, "l": 50, "r": 50, "t": 50 }, "paper_bgcolor": "white", "plot_bgcolor": "white", "showlegend": true, "xaxis": { "linecolor": "#dedede", "showgrid": false, "showline": true, "title": { "font": { "color": "black", "size": 16 } } }, "yaxis": { "gridcolor": "#dedede", "gridwidth": 1, "showgrid": true, "showline": false, "zeroline": true, "zerolinecolor": "#dedede" } } }, "title": { "text": "Air Passengers" }, "xaxis": { "title": { "text": "Month" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "FORECAST_HORIZON = 12\n", "BACKTEST_STRIDE = 12\n", "\n", "series = AirPassengersDataset().load()\n", "train, val = series[: -3 * FORECAST_HORIZON], series[-3 * FORECAST_HORIZON :]\n", "\n", "print(f\"Training: {len(train)} points | Validation: {len(val)} points\")\n", "\n", "fig = series.plotly()\n", "fig.add_vline(\n", " x=train.end_time(),\n", " line_color=\"blue\",\n", " line_dash=\"dash\",\n", " annotation_text=\"Train / val split \",\n", " annotation_position=\"top left\",\n", ")\n", "fig.update_layout(title=\"Air Passengers\", **PLOTLY_KWARGS)" ] }, { "cell_type": "markdown", "id": "4c5db7bc", "metadata": {}, "source": [ "## 3. Enable autolog\n", "\n", "One line turns on automatic experiment tracking for Darts models and metrics. We enable model logging (for the registry workflow) and backtest aggregate metrics (scalar `backtest_agg_*` keys for easy run comparison)." ] }, { "cell_type": "code", "execution_count": 5, "id": "71e43aa8", "metadata": {}, "outputs": [], "source": [ "autolog(log_models=True, log_backtest_aggregate=True)" ] }, { "cell_type": "markdown", "id": "36acc84b", "metadata": {}, "source": [ "
\n", "What does autolog capture? (click to expand)\n", "\n", "| Trigger | Logged automatically |\n", "|---|---|\n", "| `model.fit(...)` | Model tags, hyperparameters, input series info, and trained model artifact when `log_models=True` (`darts.model_is_pretrained=True`) |\n", "| `model.historical_forecasts(..., retrain=True)` | Same tags, hyperparameters, and series info as `fit()`; when `log_models=True`, also logs an untrained model template (`darts.model_is_pretrained=False`) unless a model was already logged in the run |\n", "| Standalone metric calls | Time-aggregated scalar metrics (e.g. `mae()`) and time-dependent stepped metrics (e.g. `ae()`); detailed breakdowns go to `metrics_per_series.json` |\n", "| `model.backtest(...)` | Windowed-, per-horizon-, or aggregated metrics prefixed with `backtest_`; with `log_backtest_aggregate=True` also logs `backtest_agg_{metric}` |\n", "| PyTorch models (NBEATS, TFT, …) | Per-epoch `train_loss` / `val_loss` via MLflow's PyTorch autolog |\n", "\n", "Disable anytime with `autolog(disable=True)`.\n", "\n", "For manual logging, save/load APIs, and metric-key details, see the [`Darts API reference`](https://unit8co.github.io/darts/generated_api/darts.utils.mlflow.html) and the [`MLflow API reference`](https://mlflow.org/docs/latest/ml/tracking/tracking-api/).\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "fdb06e85", "metadata": {}, "source": [ "## 4. Compare different model runs\n", "\n", "We run three distinct modelling approaches inside separate MLflow runs and compare them over the same evaluation period (backtest on rolling forecasts):\n", "\n", "1. A pre-trained global `LinearRegressionModel` that we pre-train only on the train set (`model.fit()`) and use it to produce all rolling forecasts (`retrain=False`)\n", "2. A global `LinearRegressionModel` that we re-fit for every rolling forecast (`retrain=True`).\n", "1. A local `ExponentialSmoothing` model that must be re-fit for every rolling forecast (`retrain=True`).\n", "\n", "We score the generated forecasts using `model.backtest()`:\n", "\n", "1. **Backtest** with windowed `mae` (`reduction=None` → stepped `backtest_mae` chart + scalar aggregate `backtest_agg_mae`).\n", "2. **Backtest** with time-dependent `err` (per-horizon error profile `backtest_ae` + scalar aggregate `backtest_agg_ae`)." ] }, { "cell_type": "code", "execution_count": 6, "id": "ff6030e8", "metadata": {}, "outputs": [], "source": [ "def run_experiment(name, model, pretrain, plot_historical=False):\n", " \"\"\"Fit, evaluate, and autolog a single forecasting experiment.\"\"\"\n", " hfc_kwargs = {\n", " \"series\": series,\n", " \"start\": val.start_time(), # start time of the validation period\n", " \"forecast_horizon\": FORECAST_HORIZON, # forecast horizon\n", " \"stride\": BACKTEST_STRIDE, # step size between rolling forecasts\n", " \"retrain\": not pretrain, # use pre-trained model or re-train for every forecast\n", " \"last_points_only\": False, # use all available points for each forecast\n", " }\n", "\n", " with mlflow.start_run(run_name=name):\n", " # logs pre-trained model artifact\n", " if pretrain:\n", " model.fit(train)\n", "\n", " # logs model artifact if `retrain` is used\n", " hfcs = model.historical_forecasts(**hfc_kwargs)\n", "\n", " # windowed aggregate metric: stepped backtest_mae + scalar backtest_agg_mae\n", " # note: you can also pass multiple metrics to the `metric` argument (with dedicated `metric_kwargs`)\n", " bt_kwargs = {**hfc_kwargs, \"historical_forecasts\": hfcs, \"reduction\": None}\n", " model.backtest(metric=metrics.mae, **bt_kwargs)\n", "\n", " # time-dependent metric: per-horizon error `backtest_err` + scalar backtest_agg_err\n", " model.backtest(metric=metrics.err, **bt_kwargs)\n", "\n", " if plot_historical:\n", " fig = series.plotly(label=\"actual\")\n", " for idx, hfc in enumerate(hfcs):\n", " fig = hfc.plotly(label=f\"forecast {idx}\", fig=fig)\n", " fig.update_layout(title=f\"Historical forecasts — {name}\", **PLOTLY_KWARGS)\n", " fig.show()\n", "\n", " return model" ] }, { "cell_type": "code", "execution_count": 7, "id": "12ed056e", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2026/09/04 14:03:56 WARNING mlflow.utils.environment: Failed to resolve installed pip version. ``pip`` will be added to conda.yaml environment spec without a version specifier.\n" ] }, { "data": { "application/vnd.plotly.v1+json": { "data": [ { "hovertemplate": "actual: %{y:.3g}", "legendgroup": "actual", "line": { "color": "#000000" }, "mode": "lines", "name": "actual", "type": "scatter", "x": [ "1949-01-01T00:00:00.000000000", "1949-02-01T00:00:00.000000000", "1949-03-01T00:00:00.000000000", "1949-04-01T00:00:00.000000000", "1949-05-01T00:00:00.000000000", "1949-06-01T00:00:00.000000000", "1949-07-01T00:00:00.000000000", "1949-08-01T00:00:00.000000000", "1949-09-01T00:00:00.000000000", "1949-10-01T00:00:00.000000000", "1949-11-01T00:00:00.000000000", "1949-12-01T00:00:00.000000000", "1950-01-01T00:00:00.000000000", "1950-02-01T00:00:00.000000000", "1950-03-01T00:00:00.000000000", "1950-04-01T00:00:00.000000000", "1950-05-01T00:00:00.000000000", "1950-06-01T00:00:00.000000000", "1950-07-01T00:00:00.000000000", "1950-08-01T00:00:00.000000000", "1950-09-01T00:00:00.000000000", "1950-10-01T00:00:00.000000000", "1950-11-01T00:00:00.000000000", "1950-12-01T00:00:00.000000000", "1951-01-01T00:00:00.000000000", "1951-02-01T00:00:00.000000000", "1951-03-01T00:00:00.000000000", "1951-04-01T00:00:00.000000000", "1951-05-01T00:00:00.000000000", "1951-06-01T00:00:00.000000000", "1951-07-01T00:00:00.000000000", "1951-08-01T00:00:00.000000000", "1951-09-01T00:00:00.000000000", "1951-10-01T00:00:00.000000000", "1951-11-01T00:00:00.000000000", "1951-12-01T00:00:00.000000000", "1952-01-01T00:00:00.000000000", "1952-02-01T00:00:00.000000000", "1952-03-01T00:00:00.000000000", "1952-04-01T00:00:00.000000000", "1952-05-01T00:00:00.000000000", "1952-06-01T00:00:00.000000000", "1952-07-01T00:00:00.000000000", "1952-08-01T00:00:00.000000000", "1952-09-01T00:00:00.000000000", "1952-10-01T00:00:00.000000000", "1952-11-01T00:00:00.000000000", "1952-12-01T00:00:00.000000000", "1953-01-01T00:00:00.000000000", "1953-02-01T00:00:00.000000000", "1953-03-01T00:00:00.000000000", "1953-04-01T00:00:00.000000000", "1953-05-01T00:00:00.000000000", "1953-06-01T00:00:00.000000000", "1953-07-01T00:00:00.000000000", "1953-08-01T00:00:00.000000000", "1953-09-01T00:00:00.000000000", "1953-10-01T00:00:00.000000000", "1953-11-01T00:00:00.000000000", "1953-12-01T00:00:00.000000000", "1954-01-01T00:00:00.000000000", "1954-02-01T00:00:00.000000000", "1954-03-01T00:00:00.000000000", "1954-04-01T00:00:00.000000000", "1954-05-01T00:00:00.000000000", "1954-06-01T00:00:00.000000000", "1954-07-01T00:00:00.000000000", "1954-08-01T00:00:00.000000000", "1954-09-01T00:00:00.000000000", "1954-10-01T00:00:00.000000000", "1954-11-01T00:00:00.000000000", "1954-12-01T00:00:00.000000000", "1955-01-01T00:00:00.000000000", "1955-02-01T00:00:00.000000000", "1955-03-01T00:00:00.000000000", "1955-04-01T00:00:00.000000000", "1955-05-01T00:00:00.000000000", "1955-06-01T00:00:00.000000000", "1955-07-01T00:00:00.000000000", "1955-08-01T00:00:00.000000000", "1955-09-01T00:00:00.000000000", "1955-10-01T00:00:00.000000000", "1955-11-01T00:00:00.000000000", "1955-12-01T00:00:00.000000000", "1956-01-01T00:00:00.000000000", "1956-02-01T00:00:00.000000000", "1956-03-01T00:00:00.000000000", "1956-04-01T00:00:00.000000000", "1956-05-01T00:00:00.000000000", "1956-06-01T00:00:00.000000000", "1956-07-01T00:00:00.000000000", "1956-08-01T00:00:00.000000000", "1956-09-01T00:00:00.000000000", "1956-10-01T00:00:00.000000000", "1956-11-01T00:00:00.000000000", "1956-12-01T00:00:00.000000000", "1957-01-01T00:00:00.000000000", "1957-02-01T00:00:00.000000000", "1957-03-01T00:00:00.000000000", "1957-04-01T00:00:00.000000000", "1957-05-01T00:00:00.000000000", "1957-06-01T00:00:00.000000000", "1957-07-01T00:00:00.000000000", "1957-08-01T00:00:00.000000000", "1957-09-01T00:00:00.000000000", "1957-10-01T00:00:00.000000000", "1957-11-01T00:00:00.000000000", "1957-12-01T00:00:00.000000000", "1958-01-01T00:00:00.000000000", "1958-02-01T00:00:00.000000000", "1958-03-01T00:00:00.000000000", "1958-04-01T00:00:00.000000000", "1958-05-01T00:00:00.000000000", "1958-06-01T00:00:00.000000000", "1958-07-01T00:00:00.000000000", "1958-08-01T00:00:00.000000000", "1958-09-01T00:00:00.000000000", "1958-10-01T00:00:00.000000000", "1958-11-01T00:00:00.000000000", "1958-12-01T00:00:00.000000000", "1959-01-01T00:00:00.000000000", "1959-02-01T00:00:00.000000000", "1959-03-01T00:00:00.000000000", "1959-04-01T00:00:00.000000000", "1959-05-01T00:00:00.000000000", "1959-06-01T00:00:00.000000000", "1959-07-01T00:00:00.000000000", "1959-08-01T00:00:00.000000000", "1959-09-01T00:00:00.000000000", "1959-10-01T00:00:00.000000000", "1959-11-01T00:00:00.000000000", "1959-12-01T00:00:00.000000000", "1960-01-01T00:00:00.000000000", "1960-02-01T00:00:00.000000000", "1960-03-01T00:00:00.000000000", "1960-04-01T00:00:00.000000000", "1960-05-01T00:00:00.000000000", "1960-06-01T00:00:00.000000000", "1960-07-01T00:00:00.000000000", "1960-08-01T00:00:00.000000000", "1960-09-01T00:00:00.000000000", "1960-10-01T00:00:00.000000000", "1960-11-01T00:00:00.000000000", "1960-12-01T00:00:00.000000000" ], "y": { "bdata": "AAAAAAAAXEAAAAAAAIBdQAAAAAAAgGBAAAAAAAAgYEAAAAAAAEBeQAAAAAAA4GBAAAAAAACAYkAAAAAAAIBiQAAAAAAAAGFAAAAAAADAXUAAAAAAAABaQAAAAAAAgF1AAAAAAADAXEAAAAAAAIBfQAAAAAAAoGFAAAAAAADgYEAAAAAAAEBfQAAAAAAAoGJAAAAAAABAZUAAAAAAAEBlQAAAAAAAwGNAAAAAAACgYEAAAAAAAIBcQAAAAAAAgGFAAAAAAAAgYkAAAAAAAMBiQAAAAAAAQGZAAAAAAABgZEAAAAAAAIBlQAAAAAAAQGZAAAAAAADgaEAAAAAAAOBoQAAAAAAAAGdAAAAAAABAZEAAAAAAAEBiQAAAAAAAwGRAAAAAAABgZUAAAAAAAIBmQAAAAAAAIGhAAAAAAACgZkAAAAAAAOBmQAAAAAAAQGtAAAAAAADAbEAAAAAAAEBuQAAAAAAAIGpAAAAAAADgZ0AAAAAAAIBlQAAAAAAAQGhAAAAAAACAaEAAAAAAAIBoQAAAAAAAgG1AAAAAAABgbUAAAAAAAKBsQAAAAAAAYG5AAAAAAACAcEAAAAAAAABxQAAAAAAAoG1AAAAAAABgakAAAAAAAIBmQAAAAAAAIGlAAAAAAACAaUAAAAAAAIBnQAAAAAAAYG1AAAAAAABgbEAAAAAAAEBtQAAAAAAAgHBAAAAAAADgckAAAAAAAFByQAAAAAAAMHBAAAAAAACgbEAAAAAAAGBpQAAAAAAAoGxAAAAAAABAbkAAAAAAACBtQAAAAAAAsHBAAAAAAADQcEAAAAAAAOBwQAAAAAAAsHNAAAAAAADAdkAAAAAAALB1QAAAAAAAgHNAAAAAAAAgcUAAAAAAAKBtQAAAAAAAYHFAAAAAAADAcUAAAAAAAFBxQAAAAAAA0HNAAAAAAACQc0AAAAAAAOBzQAAAAAAAYHdAAAAAAADQeUAAAAAAAFB5QAAAAAAAMHZAAAAAAAAgc0AAAAAAAPBwQAAAAAAAIHNAAAAAAACwc0AAAAAAANByQAAAAAAAQHZAAAAAAADAdUAAAAAAADB2QAAAAAAAYHpAAAAAAAAQfUAAAAAAADB9QAAAAAAAQHlAAAAAAACwdUAAAAAAABBzQAAAAAAAAHVAAAAAAABAdUAAAAAAAOBzQAAAAAAAoHZAAAAAAADAdUAAAAAAALB2QAAAAAAAMHtAAAAAAACwfkAAAAAAAJB/QAAAAAAAQHlAAAAAAABwdkAAAAAAAGBzQAAAAAAAEHVAAAAAAACAdkAAAAAAAGB1QAAAAAAAYHlAAAAAAADAeEAAAAAAAEB6QAAAAAAAgH1AAAAAAAAggUAAAAAAAHiBQAAAAAAA8HxAAAAAAABweUAAAAAAAKB2QAAAAAAAUHlAAAAAAAAQekAAAAAAAHB4QAAAAAAAMHpAAAAAAADQfEAAAAAAAIB9QAAAAAAAuIBAAAAAAABwg0AAAAAAAPCCQAAAAAAAwH9AAAAAAADQfEAAAAAAAGB4QAAAAAAAAHtA", "dtype": "f8" } }, { "hovertemplate": "forecast 0: %{y:.3g}", "legendgroup": "forecast 0", "line": { "color": "#003dfd" }, "mode": "lines", "name": "forecast 0", "type": "scatter", "x": [ "1958-01-01T00:00:00.000000000", "1958-02-01T00:00:00.000000000", "1958-03-01T00:00:00.000000000", "1958-04-01T00:00:00.000000000", "1958-05-01T00:00:00.000000000", "1958-06-01T00:00:00.000000000", "1958-07-01T00:00:00.000000000", "1958-08-01T00:00:00.000000000", "1958-09-01T00:00:00.000000000", "1958-10-01T00:00:00.000000000", "1958-11-01T00:00:00.000000000", "1958-12-01T00:00:00.000000000" ], "y": { "bdata": "DDQnI6QgdkBKYFmmDEZ1QNoNvZcioHhAIcYwrxV3eEDGoXpXwfh4QFCjzLQ19X1AjtbAxQprgEDQdLHYvXeAQBampaePpXxA/N2iqG9MeEBQFfttPoF1QLKTcZKhI3dA", "dtype": "f8" } }, { "hovertemplate": "forecast 1: %{y:.3g}", "legendgroup": "forecast 1", "line": { "color": "#b512b8" }, "mode": "lines", "name": "forecast 1", "type": "scatter", "x": [ "1959-01-01T00:00:00.000000000", "1959-02-01T00:00:00.000000000", "1959-03-01T00:00:00.000000000", "1959-04-01T00:00:00.000000000", "1959-05-01T00:00:00.000000000", "1959-06-01T00:00:00.000000000", "1959-07-01T00:00:00.000000000", "1959-08-01T00:00:00.000000000", "1959-09-01T00:00:00.000000000", "1959-10-01T00:00:00.000000000", "1959-11-01T00:00:00.000000000", "1959-12-01T00:00:00.000000000" ], "y": { "bdata": "tglrg62hd0DcWMd7ADN2QMGurWZVy3hAZe8PRfpSeEAiIIv+AWh5QBIw56L3sn5AebRxL4NcgUCvuEOEs6qBQP5NWzXs03xAbBZl4Nj2eEBzPKKqWg12QGjR+vqpOHdA", "dtype": "f8" } }, { "hovertemplate": "forecast 2: %{y:.3g}", "legendgroup": "forecast 2", "line": { "color": "#11a9ba" }, "mode": "lines", "name": "forecast 2", "type": "scatter", "x": [ "1960-01-01T00:00:00.000000000", "1960-02-01T00:00:00.000000000", "1960-03-01T00:00:00.000000000", "1960-04-01T00:00:00.000000000", "1960-05-01T00:00:00.000000000", "1960-06-01T00:00:00.000000000", "1960-07-01T00:00:00.000000000", "1960-08-01T00:00:00.000000000", "1960-09-01T00:00:00.000000000", "1960-10-01T00:00:00.000000000", "1960-11-01T00:00:00.000000000", "1960-12-01T00:00:00.000000000" ], "y": { "bdata": "4tWOkrWzeUCRUBweMXd4QO7LYWVTInxAobH0G48UfECQ9sMYrHJ9QFfj/mwc5IBA+LMuueVEg0AqcP9o8LaDQBC2RYNeYIBAYNmBHBZzfEBkyez9rWN5QNIWZm7fkXtA", "dtype": "f8" } } ], "layout": { "hovermode": "x unified", "legend": { "x": 0.01, "xanchor": "left", "y": 0.99, "yanchor": "top" }, "template": { "data": { "scatter": [ { "line": { "width": 3 }, "type": "scatter" } ] }, "layout": { "colorway": [ "#000000", "#003dfd", "#b512b8", "#11a9ba", "#0d780f", "#f77f07", "#ba0f0f" ], "font": { "color": "black", "family": "Arial, sans-serif", "size": 14 }, "legend": { "bgcolor": "rgba(255, 255, 255, 0.8)", "borderwidth": 0, "font": { "size": 14 }, "x": 1, "xanchor": "right", "y": 1, "yanchor": "top" }, "margin": { "b": 50, "l": 50, "r": 50, "t": 50 }, "paper_bgcolor": "white", "plot_bgcolor": "white", "showlegend": true, "xaxis": { "linecolor": "#dedede", "showgrid": false, "showline": true, "title": { "font": { "color": "black", "size": 16 } } }, "yaxis": { "gridcolor": "#dedede", "gridwidth": 1, "showgrid": true, "showline": false, "zeroline": true, "zerolinecolor": "#dedede" } } }, "title": { "text": "Historical forecasts — linear_regression_pretrained" }, "xaxis": { "title": { "text": "Month" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stderr", "output_type": "stream", "text": [ "2026/09/04 14:03:56 WARNING mlflow.utils.environment: Failed to resolve installed pip version. ``pip`` will be added to conda.yaml environment spec without a version specifier.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Finished run: linear_regression_pretrained\n", "Finished run: linear_regression\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2026/09/04 14:03:56 WARNING mlflow.utils.environment: Failed to resolve installed pip version. ``pip`` will be added to conda.yaml environment spec without a version specifier.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Finished run: exponential_smoothing\n" ] } ], "source": [ "run_configs = [\n", " {\n", " \"name\": \"linear_regression_pretrained\",\n", " \"model\": LinearRegressionModel(lags=12, output_chunk_length=FORECAST_HORIZON),\n", " \"pretrain\": True,\n", " \"plot_historical\": True,\n", " },\n", " {\n", " \"name\": \"linear_regression\",\n", " \"model\": LinearRegressionModel(lags=12, output_chunk_length=FORECAST_HORIZON),\n", " \"pretrain\": False,\n", " \"plot_historical\": False,\n", " },\n", " {\n", " \"name\": \"exponential_smoothing\",\n", " \"model\": ExponentialSmoothing(),\n", " \"pretrain\": False,\n", " \"plot_historical\": False,\n", " },\n", "]\n", "\n", "for run_config in run_configs:\n", " run_experiment(**run_config)\n", " print(f\"Finished run: {run_config['name']}\")" ] }, { "cell_type": "markdown", "id": "7b04a702", "metadata": {}, "source": [ "### Disable Autologging\n", "\n", "You can turn off autologging at any time with the following line." ] }, { "cell_type": "code", "execution_count": 8, "id": "893979c6", "metadata": {}, "outputs": [], "source": [ "autolog(disable=True)" ] }, { "cell_type": "markdown", "id": "b3221514", "metadata": {}, "source": [ "### Explore in the MLflow UI\n", "\n", "Open the tracking UI and compare runs side by side — model parameters, metrics, model artifacts are all available automatically." ] }, { "cell_type": "code", "execution_count": 9, "id": "ed4e0c35", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "To explore runs in the UI:\n", " mlflow ui --backend-store-uri sqlite:////var/folders/4h/l09drklx06g8022q7khgyxyr0000gn/T/tmpjnang74m/mlflow.db\n" ] } ], "source": [ "print(\n", " f\"\\nTo explore runs in the UI:\\n mlflow ui --backend-store-uri sqlite:///{mlflow_db}\"\n", ")" ] }, { "cell_type": "markdown", "id": "834d1825", "metadata": {}, "source": [ "
\n", "MLflow Runs Table View\n", "\n", "The MLflow Runs Table shows all runs under an experiment in tabular style. It allows you to filter, sort, compare anything that was logged. You can add any logged parameter (metrics, model parameters, ...) to the table via the `Columns` dropdown.\n", "\n", "Here we see that:\n", "\n", "- `linear_regression_pretrained` had the lowest aggregated MAE (`backtest_agg_mae`).\n", "- `exponential_smoothing` had the lowest aggregated horizon-based bias (`backtest_agg_err`).\n", "\n", "![MLflow runs table view](./static/images/mlflow_experiments_overview.png)\n", "\n", "
\n", "\n", "
\n", "MLflow Runs Chart View\n", "\n", "Click on the `Chart View` icon on the top left of the Runs Table to show a detailed view of all metrics across runs. In this view you can find scalar as well as stepped metric charts.\n", "\n", "- 📈 Stepped metrics (line charts): \n", " - Windowed-backtest results for [time-aggregated metrics](https://unit8co.github.io/darts/generated_api/darts.metrics.html) - showing the score per rolling forecast (0, 1, ... number of rolling windows - 1). See `backtest_mae` below showing 3 steps corresponding to the 3 rolling historical forecasts. \n", " - Horizon-based-backtest results for [time-dependent metrics](https://unit8co.github.io/darts/generated_api/darts.metrics.html) - showing the score per step in the forecast horizon (0, 1, ... horizon - 1) aggregated over all rolling forecasts. See `backtest_err` below showing 12 steps corresponding to `FORECAST_HORIZON`.\n", "- 📊 Scalar metrics (bar charts) for: \n", " - Aggregated backtest metrics, e.g. with `autolog(log_backtest_aggregate=True)`. See `backtest_agg_mae` and `backtest_agg_ae` below.\n", " - Direct [time-aggregated metric](https://unit8co.github.io/darts/generated_api/darts.metrics.html) calls (e.g. `darts.metrics.mae()`)\n", "\n", "... and many other configurations. Read more in the [API reference](https://unit8co.github.io/darts/generated_api/darts.utils.mlflow.html).\n", "\n", "![MLflow runs chart view](./static/images/mlflow_experiments_metrics.png)\n", "\n", "
\n", "\n", "
\n", "MLflow Run Detail Page (click to expand)\n", "\n", "Click on any run in the Runs Table to open the Run Detail Page. It shows an overview of the run, its recorded metrics, hyper-parameters, input data information, tags, and more. Play around with the UI to see the different views and features.\n", "\n", "Scroll down to the \"Model\" section and you will see the model that was logged during training. Click on the model to view the details (inclduing the logged model artifacts).\n", "\n", "![MLflow run detail page](./static/images/mlflow_run_detail.png)\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "248f15dc", "metadata": {}, "source": [ "## 5. Find the best run\n", "\n", "Above, we could see that `linear_regression_pretrained` had the lowest MAE.\n", "\n", "Let's do this once programmatically. We want to find the model with the lowest overall MAE. With `log_backtest_aggregate=True` we get a convient sort key: one scalar that summarizes rolling backtest performance, regardless of how many windows or time series were evaluated. `backtest_agg_mae` is such a key." ] }, { "cell_type": "code", "execution_count": 10, "id": "fa88b420", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
run_idtags.mlflow.runNamemetrics.backtest_agg_mae
0ee57c5714d6442ce93fbcf9899968842linear_regression_pretrained18.870433
10baa4c6c0e4d4ad5b7b53f5ae887ba02exponential_smoothing19.609498
28869add4ede74eebae1570acae7d82c6linear_regression22.533742
\n", "
" ], "text/plain": [ " run_id tags.mlflow.runName \\\n", "0 ee57c5714d6442ce93fbcf9899968842 linear_regression_pretrained \n", "1 0baa4c6c0e4d4ad5b7b53f5ae887ba02 exponential_smoothing \n", "2 8869add4ede74eebae1570acae7d82c6 linear_regression \n", "\n", " metrics.backtest_agg_mae \n", "0 18.870433 \n", "1 19.609498 \n", "2 22.533742 " ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Best run by backtest_agg_mae: linear_regression_pretrained (ee57c5714d6442ce93fbcf9899968842)\n" ] } ], "source": [ "runs_df = mlflow.search_runs(\n", " experiment_names=[EXPERIMENT_NAME],\n", " order_by=[\"metrics.backtest_agg_mae ASC\"],\n", ")\n", "display(runs_df[[\"run_id\", \"tags.mlflow.runName\", \"metrics.backtest_agg_mae\"]])\n", "\n", "best_run = runs_df.iloc[0]\n", "best_run_id = best_run.run_id\n", "best_run_name = best_run[\"tags.mlflow.runName\"]\n", "print(f\"\\nBest run by backtest_agg_mae: {best_run_name} ({best_run_id})\")" ] }, { "cell_type": "markdown", "id": "f4606742", "metadata": {}, "source": [ "## 6. Register the champion model\n", "\n", "We can promote the best run's model to the **MLflow Model Registry**, then assign a **champion** alias (or any other alias). Downstream code can then load the registered models via registered model name and alias.\n", "\n", "> This assumes that model logging was enabled using `autolog(log_models=True)`:\n", "> - runs that call `fit()` within the run store a pre-trained model\n", "> - runs that only use `historical_forecasts(retrain=True)` store an untrained template (check `darts.model_is_pretrained` on the run or model metadata)." ] }, { "cell_type": "code", "execution_count": 11, "id": "26ca96da", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Best model URI: models:/m-f25a0591479a43d085e31accbe45cc4a\n", "Aliases set: @champion\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Successfully registered model 'darts-air-passengers-forecaster'.\n", "Created version '1' of model 'darts-air-passengers-forecaster'.\n" ] } ], "source": [ "# Get the logged model URI of the best run\n", "model_outputs = mlflow.get_run(best_run_id).outputs.model_outputs\n", "best_model_uri = f\"models:/{model_outputs[0].model_id}\"\n", "print(f\"Best model URI: {best_model_uri}\")\n", "\n", "REGISTERED_MODEL_NAME = \"darts-air-passengers-forecaster\"\n", "\n", "registration = mlflow.register_model(\n", " model_uri=best_model_uri,\n", " name=REGISTERED_MODEL_NAME,\n", ")\n", "\n", "client = MlflowClient()\n", "client.set_registered_model_alias(\n", " REGISTERED_MODEL_NAME, \"champion\", registration.version\n", ")\n", "print(\"Aliases set: @champion\")" ] }, { "cell_type": "markdown", "id": "b7abaf63", "metadata": {}, "source": [ "All logged models can be found in the MLflow Models View:\n", "\n", "![MLflow Model Table](./static/images/mlflow_models_overview.png)\n", "\n", "The registered models can be found in the MLflow Model Registry:\n", "\n", "![MLflow Model Registry](./static/images/mlflow_model_registry.png)" ] }, { "cell_type": "markdown", "id": "5cd2ebab", "metadata": {}, "source": [ "## 7. Load the champion and forecast\n", "\n", "Production inference loads the aliased registered model via `models:/@champion` without hard-coding a version number.\n", "The loaded model can then be used for prediction.\n", "\n", "We use `model.historical_forecasts()` with `start=\"end\"` (instead of `model.predict()`) to produce the final forecasts because it also handles potential re-training for local and non-pre-trained models.\n", "\n", "> Always load Darts models with `from darts.utils.mlflow import load_model` — not `mlflow.pyfunc.load_model`." ] }, { "cell_type": "code", "execution_count": 12, "id": "c12ba777", "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "data": [ { "hovertemplate": "full series: %{y:.3g}", "legendgroup": "full series", "line": { "color": "#000000" }, "mode": "lines", "name": "full series", "type": "scatter", "x": [ "1949-01-01T00:00:00.000000000", "1949-02-01T00:00:00.000000000", "1949-03-01T00:00:00.000000000", "1949-04-01T00:00:00.000000000", "1949-05-01T00:00:00.000000000", "1949-06-01T00:00:00.000000000", "1949-07-01T00:00:00.000000000", "1949-08-01T00:00:00.000000000", "1949-09-01T00:00:00.000000000", "1949-10-01T00:00:00.000000000", "1949-11-01T00:00:00.000000000", "1949-12-01T00:00:00.000000000", "1950-01-01T00:00:00.000000000", "1950-02-01T00:00:00.000000000", "1950-03-01T00:00:00.000000000", "1950-04-01T00:00:00.000000000", "1950-05-01T00:00:00.000000000", "1950-06-01T00:00:00.000000000", "1950-07-01T00:00:00.000000000", "1950-08-01T00:00:00.000000000", "1950-09-01T00:00:00.000000000", "1950-10-01T00:00:00.000000000", "1950-11-01T00:00:00.000000000", "1950-12-01T00:00:00.000000000", "1951-01-01T00:00:00.000000000", "1951-02-01T00:00:00.000000000", "1951-03-01T00:00:00.000000000", "1951-04-01T00:00:00.000000000", "1951-05-01T00:00:00.000000000", "1951-06-01T00:00:00.000000000", "1951-07-01T00:00:00.000000000", "1951-08-01T00:00:00.000000000", "1951-09-01T00:00:00.000000000", "1951-10-01T00:00:00.000000000", "1951-11-01T00:00:00.000000000", "1951-12-01T00:00:00.000000000", "1952-01-01T00:00:00.000000000", "1952-02-01T00:00:00.000000000", "1952-03-01T00:00:00.000000000", "1952-04-01T00:00:00.000000000", "1952-05-01T00:00:00.000000000", "1952-06-01T00:00:00.000000000", "1952-07-01T00:00:00.000000000", "1952-08-01T00:00:00.000000000", "1952-09-01T00:00:00.000000000", "1952-10-01T00:00:00.000000000", "1952-11-01T00:00:00.000000000", "1952-12-01T00:00:00.000000000", "1953-01-01T00:00:00.000000000", "1953-02-01T00:00:00.000000000", "1953-03-01T00:00:00.000000000", "1953-04-01T00:00:00.000000000", "1953-05-01T00:00:00.000000000", "1953-06-01T00:00:00.000000000", "1953-07-01T00:00:00.000000000", "1953-08-01T00:00:00.000000000", "1953-09-01T00:00:00.000000000", "1953-10-01T00:00:00.000000000", "1953-11-01T00:00:00.000000000", "1953-12-01T00:00:00.000000000", "1954-01-01T00:00:00.000000000", "1954-02-01T00:00:00.000000000", "1954-03-01T00:00:00.000000000", "1954-04-01T00:00:00.000000000", "1954-05-01T00:00:00.000000000", "1954-06-01T00:00:00.000000000", "1954-07-01T00:00:00.000000000", "1954-08-01T00:00:00.000000000", "1954-09-01T00:00:00.000000000", "1954-10-01T00:00:00.000000000", "1954-11-01T00:00:00.000000000", "1954-12-01T00:00:00.000000000", "1955-01-01T00:00:00.000000000", "1955-02-01T00:00:00.000000000", "1955-03-01T00:00:00.000000000", "1955-04-01T00:00:00.000000000", "1955-05-01T00:00:00.000000000", "1955-06-01T00:00:00.000000000", "1955-07-01T00:00:00.000000000", "1955-08-01T00:00:00.000000000", "1955-09-01T00:00:00.000000000", "1955-10-01T00:00:00.000000000", "1955-11-01T00:00:00.000000000", "1955-12-01T00:00:00.000000000", "1956-01-01T00:00:00.000000000", "1956-02-01T00:00:00.000000000", "1956-03-01T00:00:00.000000000", "1956-04-01T00:00:00.000000000", "1956-05-01T00:00:00.000000000", "1956-06-01T00:00:00.000000000", "1956-07-01T00:00:00.000000000", "1956-08-01T00:00:00.000000000", "1956-09-01T00:00:00.000000000", "1956-10-01T00:00:00.000000000", "1956-11-01T00:00:00.000000000", "1956-12-01T00:00:00.000000000", "1957-01-01T00:00:00.000000000", "1957-02-01T00:00:00.000000000", "1957-03-01T00:00:00.000000000", "1957-04-01T00:00:00.000000000", "1957-05-01T00:00:00.000000000", "1957-06-01T00:00:00.000000000", "1957-07-01T00:00:00.000000000", "1957-08-01T00:00:00.000000000", "1957-09-01T00:00:00.000000000", "1957-10-01T00:00:00.000000000", "1957-11-01T00:00:00.000000000", "1957-12-01T00:00:00.000000000", "1958-01-01T00:00:00.000000000", "1958-02-01T00:00:00.000000000", "1958-03-01T00:00:00.000000000", "1958-04-01T00:00:00.000000000", "1958-05-01T00:00:00.000000000", "1958-06-01T00:00:00.000000000", "1958-07-01T00:00:00.000000000", "1958-08-01T00:00:00.000000000", "1958-09-01T00:00:00.000000000", "1958-10-01T00:00:00.000000000", "1958-11-01T00:00:00.000000000", "1958-12-01T00:00:00.000000000", "1959-01-01T00:00:00.000000000", "1959-02-01T00:00:00.000000000", "1959-03-01T00:00:00.000000000", "1959-04-01T00:00:00.000000000", "1959-05-01T00:00:00.000000000", "1959-06-01T00:00:00.000000000", "1959-07-01T00:00:00.000000000", "1959-08-01T00:00:00.000000000", "1959-09-01T00:00:00.000000000", "1959-10-01T00:00:00.000000000", "1959-11-01T00:00:00.000000000", "1959-12-01T00:00:00.000000000", "1960-01-01T00:00:00.000000000", "1960-02-01T00:00:00.000000000", "1960-03-01T00:00:00.000000000", "1960-04-01T00:00:00.000000000", "1960-05-01T00:00:00.000000000", "1960-06-01T00:00:00.000000000", "1960-07-01T00:00:00.000000000", "1960-08-01T00:00:00.000000000", "1960-09-01T00:00:00.000000000", "1960-10-01T00:00:00.000000000", "1960-11-01T00:00:00.000000000", "1960-12-01T00:00:00.000000000" ], "y": { "bdata": "AAAAAAAAXEAAAAAAAIBdQAAAAAAAgGBAAAAAAAAgYEAAAAAAAEBeQAAAAAAA4GBAAAAAAACAYkAAAAAAAIBiQAAAAAAAAGFAAAAAAADAXUAAAAAAAABaQAAAAAAAgF1AAAAAAADAXEAAAAAAAIBfQAAAAAAAoGFAAAAAAADgYEAAAAAAAEBfQAAAAAAAoGJAAAAAAABAZUAAAAAAAEBlQAAAAAAAwGNAAAAAAACgYEAAAAAAAIBcQAAAAAAAgGFAAAAAAAAgYkAAAAAAAMBiQAAAAAAAQGZAAAAAAABgZEAAAAAAAIBlQAAAAAAAQGZAAAAAAADgaEAAAAAAAOBoQAAAAAAAAGdAAAAAAABAZEAAAAAAAEBiQAAAAAAAwGRAAAAAAABgZUAAAAAAAIBmQAAAAAAAIGhAAAAAAACgZkAAAAAAAOBmQAAAAAAAQGtAAAAAAADAbEAAAAAAAEBuQAAAAAAAIGpAAAAAAADgZ0AAAAAAAIBlQAAAAAAAQGhAAAAAAACAaEAAAAAAAIBoQAAAAAAAgG1AAAAAAABgbUAAAAAAAKBsQAAAAAAAYG5AAAAAAACAcEAAAAAAAABxQAAAAAAAoG1AAAAAAABgakAAAAAAAIBmQAAAAAAAIGlAAAAAAACAaUAAAAAAAIBnQAAAAAAAYG1AAAAAAABgbEAAAAAAAEBtQAAAAAAAgHBAAAAAAADgckAAAAAAAFByQAAAAAAAMHBAAAAAAACgbEAAAAAAAGBpQAAAAAAAoGxAAAAAAABAbkAAAAAAACBtQAAAAAAAsHBAAAAAAADQcEAAAAAAAOBwQAAAAAAAsHNAAAAAAADAdkAAAAAAALB1QAAAAAAAgHNAAAAAAAAgcUAAAAAAAKBtQAAAAAAAYHFAAAAAAADAcUAAAAAAAFBxQAAAAAAA0HNAAAAAAACQc0AAAAAAAOBzQAAAAAAAYHdAAAAAAADQeUAAAAAAAFB5QAAAAAAAMHZAAAAAAAAgc0AAAAAAAPBwQAAAAAAAIHNAAAAAAACwc0AAAAAAANByQAAAAAAAQHZAAAAAAADAdUAAAAAAADB2QAAAAAAAYHpAAAAAAAAQfUAAAAAAADB9QAAAAAAAQHlAAAAAAACwdUAAAAAAABBzQAAAAAAAAHVAAAAAAABAdUAAAAAAAOBzQAAAAAAAoHZAAAAAAADAdUAAAAAAALB2QAAAAAAAMHtAAAAAAACwfkAAAAAAAJB/QAAAAAAAQHlAAAAAAABwdkAAAAAAAGBzQAAAAAAAEHVAAAAAAACAdkAAAAAAAGB1QAAAAAAAYHlAAAAAAADAeEAAAAAAAEB6QAAAAAAAgH1AAAAAAAAggUAAAAAAAHiBQAAAAAAA8HxAAAAAAABweUAAAAAAAKB2QAAAAAAAUHlAAAAAAAAQekAAAAAAAHB4QAAAAAAAMHpAAAAAAADQfEAAAAAAAIB9QAAAAAAAuIBAAAAAAABwg0AAAAAAAPCCQAAAAAAAwH9AAAAAAADQfEAAAAAAAGB4QAAAAAAAAHtA", "dtype": "f8" } }, { "hovertemplate": "champion forecast: %{y:.3g}", "legendgroup": "champion forecast", "line": { "color": "#003dfd" }, "mode": "lines", "name": "champion forecast", "type": "scatter", "x": [ "1961-01-01T00:00:00.000000000", "1961-02-01T00:00:00.000000000", "1961-03-01T00:00:00.000000000", "1961-04-01T00:00:00.000000000", "1961-05-01T00:00:00.000000000", "1961-06-01T00:00:00.000000000", "1961-07-01T00:00:00.000000000", "1961-08-01T00:00:00.000000000", "1961-09-01T00:00:00.000000000", "1961-10-01T00:00:00.000000000", "1961-11-01T00:00:00.000000000", "1961-12-01T00:00:00.000000000" ], "y": { "bdata": "apyip5ulfUA/xDkvmm97QP4XzmdWZ31AQE6TQYUXgEC2hog6wqyAQOfa9FT95YJAN3ns9M/4hUAandSvvzGFQHbuCyuuCIJAPJUZFfPuf0Bb3B5nYmh7QIoI0QyhXH1A", "dtype": "f8" } } ], "layout": { "hovermode": "x unified", "legend": { "x": 0.01, "xanchor": "left", "y": 0.99, "yanchor": "top" }, "template": { "data": { "scatter": [ { "line": { "width": 3 }, "type": "scatter" } ] }, "layout": { "colorway": [ "#000000", "#003dfd", "#b512b8", "#11a9ba", "#0d780f", "#f77f07", "#ba0f0f" ], "font": { "color": "black", "family": "Arial, sans-serif", "size": 14 }, "legend": { "bgcolor": "rgba(255, 255, 255, 0.8)", "borderwidth": 0, "font": { "size": 14 }, "x": 1, "xanchor": "right", "y": 1, "yanchor": "top" }, "margin": { "b": 50, "l": 50, "r": 50, "t": 50 }, "paper_bgcolor": "white", "plot_bgcolor": "white", "showlegend": true, "xaxis": { "linecolor": "#dedede", "showgrid": false, "showline": true, "title": { "font": { "color": "black", "size": 16 } } }, "yaxis": { "gridcolor": "#dedede", "gridwidth": 1, "showgrid": true, "showline": false, "zeroline": true, "zerolinecolor": "#dedede" } } }, "title": { "text": "Production inference — darts-air-passengers-forecaster@champion" }, "xaxis": { "title": { "text": "Month" } } } }, "text/html": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "champion = load_model(f\"models:/{REGISTERED_MODEL_NAME}@champion\")\n", "\n", "forecast = champion.historical_forecasts(\n", " start=\"end\", # forward-looking forecast\n", " series=series,\n", " forecast_horizon=FORECAST_HORIZON,\n", " retrain=not champion._fit_called,\n", " last_points_only=False,\n", " overlap_end=True,\n", ")[0]\n", "\n", "fig = series.plotly(label=\"full series\")\n", "fig = forecast.plotly(label=\"champion forecast\", fig=fig)\n", "fig.update_layout(\n", " title=f\"Production inference — {REGISTERED_MODEL_NAME}@champion\", **PLOTLY_KWARGS\n", ")" ] }, { "cell_type": "markdown", "id": "ac44a0ae", "metadata": {}, "source": [ "## Summary\n", "\n", "In a few lines of code you get a complete experimentation loop:\n", "\n", "1. **Setup** — tracking URI + experiment.\n", "2. **Autolog** — `autolog(log_models=True, log_backtest_aggregate=True)`.\n", "3. **Experiment** — `with mlflow.start_run()`: fit, historical forecasts, backtest metrics.\n", "4. **Compare** — `mlflow.search_runs(..., order_by=[\"metrics.backtest_agg_mae ASC\"])`.\n", "5. **Promote** — `register_model` + `set_registered_model_alias(..., \"champion\", ...)`.\n", "6. **Serve** — `load_model(\"models:/@champion\").predict(...)`.\n", "\n", "**Learn more:**\n", "\n", "- [Darts MLflow API reference](https://unit8co.github.io/darts/generated_api/darts.utils.mlflow.html)\n", "- [MLflow API reference](https://mlflow.org/docs/latest/ml/tracking/tracking-api/)\n" ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.7" } }, "nbformat": 4, "nbformat_minor": 5 }