Distributed Forecast

This interface is only tested on Linux

`DistributedMLForecast`

DistributedMLForecast(
    models,
    freq,
    lags=None,
    lag_transforms=None,
    date_features=None,
    num_threads=1,
    target_transforms=None,
    engine=None,
    num_partitions=None,
    lag_transforms_namer=None,
)

Multi backend distributed pipeline Create distributed forecast object Parameters:

Name	Type	Description	Default
`models`	`regressor or list of regressors`	Models that will be trained and used to compute the forecasts.	required
`freq`	`str or int`	Pandas offset alias, e.g. ‘D’, ‘W-THU’ or integer denoting the frequency of the series. Defaults to None.	required
`lags`	`list of int`	Lags of the target to use as features. Defaults to None.	`None`
`lag_transforms`	`dict of int to list of functions`	Mapping of target lags to their transformations. Defaults to None.	`None`
`date_features`	`list of str or callable`	Features computed from the dates. Can be pandas date attributes or functions that will take the dates as input. Defaults to None.	`None`
`num_threads`	`int`	Number of threads to use when computing the features. Use -1 to use all available CPU cores. Defaults to 1.	`1`
`target_transforms`	`list of transformers`	Transformations that will be applied to the target before computing the features and restored after the forecasting step. Defaults to None.	`None`
`engine`	`fugue execution engine`	Dask Client, Spark Session, etc to use for the distributed computation. If None will infer depending on the input type. Defaults to None.	`None`
`num_partitions`	`number of data partitions to use`	If None, the default partitions provided by the AnyDataFrame used by the `fit` and `cross_validation` methods will be used. If a Ray Dataset is provided and `num_partitions` is None, the partitioning will be done by the `id_col`. Defaults to None.	`None`
`lag_transforms_namer`	`callable`	Function that takes a transformation (either function or class), a lag and extra arguments and produces a name. Defaults to None.	`None`

`DistributedMLForecast.fit`

fit(
    df,
    id_col="unique_id",
    time_col="ds",
    target_col="y",
    static_features=None,
    dropna=True,
    keep_last_n=None,
    weight_col=None,
)

Apply the feature engineering and train the models. Parameters:

Name	Type	Description	Default
`df`	`dask, spark or ray DataFrame`	Series data in long format.	required
`id_col`	`str`	Column that identifies each serie. Defaults to ‘unique_id’.	`‘unique_id’`
`time_col`	`str`	Column that identifies each timestep, its values can be timestamps or integers. Defaults to ‘ds’.	`‘ds’`
`target_col`	`str`	Column that contains the target. Defaults to ‘y’.	`‘y’`
`static_features`	`list of str`	Names of the features that are static and will be repeated when forecasting. Defaults to None.	`None`
`dropna`	`bool`	Drop rows with missing values produced by the transformations. Defaults to True.	`True`
`keep_last_n`	`int`	Keep only these many records from each serie for the forecasting step. Can save time and memory if your features allow it. Defaults to None.	`None`
`weight_col`	`str`	Column that contains the sample weights. Defaults to None.	`None`

Returns:

Type	Description
`DistributedMLForecast`	Forecast object with series values and trained models.

`DistributedMLForecast.predict`

predict(
    h,
    before_predict_callback=None,
    after_predict_callback=None,
    X_df=None,
    new_df=None,
    ids=None,
)

Compute the predictions for the next horizon steps. Parameters:

Name	Type	Description	Default
`h`	`int`	Forecast horizon.	required
`before_predict_callback`	`callable`	Function to call on the features before computing the predictions. This function will take the input dataframe that will be passed to the model for predicting and should return a dataframe with the same structure. The series identifier is on the index. Defaults to None.	`None`
`after_predict_callback`	`callable`	Function to call on the predictions before updating the targets. This function will take a pandas Series with the predictions and should return another one with the same structure. The series identifier is on the index. Defaults to None.	`None`
`X_df`	`pandas DataFrame`	Dataframe with the future exogenous features. Should have the id column and the time column. Defaults to None.	`None`
`new_df`	`dask or spark DataFrame`	Series data of new observations for which forecasts are to be generated. This dataframe should have the same structure as the one used to fit the model, including any features and time series data. If `new_df` is not None, the method will generate forecasts for the new observations. Defaults to None.	`None`
`ids`	`list of str`	List with subset of ids seen during training for which the forecasts should be computed. Defaults to None.	`None`

Returns:

Type	Description
`dask, spark or ray DataFrame`	Predictions for each serie and timestep, with one column per model.

`DistributedMLForecast.save`

save(path)

Save forecast object Parameters:

Name	Type	Description	Default
`path`	`str`	Directory where artifacts will be stored.	required

`DistributedMLForecast.load`

load(path, engine)

Load forecast object Parameters:

Name	Type	Description	Default
`path`	`str`	Directory with saved artifacts.	required
`engine`	`fugue execution engine`	Dask Client, Spark Session, etc to use for the distributed computation.	required

`DistributedMLForecast.update`

update(df)

Update the values of the stored series. Parameters:

Name	Type	Description	Default
`df`	`pandas DataFrame`	Dataframe with new observations.	required

`DistributedMLForecast.to_local`

to_local()

Convert this distributed forecast object into a local one This pulls all the data from the remote machines, so you have to be sure that it fits in the scheduler/driver. If you’re not sure use the save method instead. Returns:

Type	Description
`MLForecast`	Local forecast object.

`DistributedMLForecast.preprocess`

preprocess(
    df,
    id_col="unique_id",
    time_col="ds",
    target_col="y",
    static_features=None,
    dropna=True,
    keep_last_n=None,
)

Add the features to data. Parameters:

Name	Type	Description	Default
`df`	`dask, spark or ray DataFrame`	Series data in long format.	required
`id_col`	`str`	Column that identifies each serie. Defaults to ‘unique_id’.	`‘unique_id’`
`time_col`	`str`	Column that identifies each timestep, its values can be timestamps or integers. Defaults to ‘ds’.	`‘ds’`
`target_col`	`str`	Column that contains the target. Defaults to ‘y’.	`‘y’`
`static_features`	`list of str`	Names of the features that are static and will be repeated when forecasting. Defaults to None.	`None`
`dropna`	`bool`	Drop rows with missing values produced by the transformations. Defaults to True.	`True`
`keep_last_n`	`int`	Keep only these many records from each serie for the forecasting step. Can save time and memory if your features allow it. Defaults to None.	`None`

Returns:

Type	Description
`same type as df`	`df` with added features.

`DistributedMLForecast.cross_validation`

cross_validation(
    df,
    n_windows,
    h,
    id_col="unique_id",
    time_col="ds",
    target_col="y",
    step_size=None,
    static_features=None,
    dropna=True,
    keep_last_n=None,
    refit=True,
    before_predict_callback=None,
    after_predict_callback=None,
    input_size=None,
    weight_col=None,
)

Perform time series cross validation. Creates n_windows splits where each window has h test periods, trains the models, computes the predictions and merges the actuals. Parameters:

Name	Type	Description	Default
`df`	`dask, spark or ray DataFrame`	Series data in long format.	required
`n_windows`	`int`	Number of windows to evaluate.	required
`h`	`int`	Number of test periods in each window.	required
`id_col`	`str`	Column that identifies each serie. Defaults to ‘unique_id’.	`‘unique_id’`
`time_col`	`str`	Column that identifies each timestep, its values can be timestamps or integers. Defaults to ‘ds’.	`‘ds’`
`target_col`	`str`	Column that contains the target. Defaults to ‘y’.	`‘y’`
`step_size`	`int`	Step size between each cross validation window. If None it will be equal to `h`. Defaults to None.	`None`
`static_features`	`list of str`	Names of the features that are static and will be repeated when forecasting. Defaults to None.	`None`
`dropna`	`bool`	Drop rows with missing values produced by the transformations. Defaults to True.	`True`
`keep_last_n`	`int`	Keep only these many records from each serie for the forecasting step. Can save time and memory if your features allow it. Defaults to None.	`None`
`refit`	`bool`	Retrain model for each cross validation window. If False, the models are trained at the beginning and then used to predict each window. Defaults to True.	`True`
`before_predict_callback`	`callable`	Function to call on the features before computing the predictions. This function will take the input dataframe that will be passed to the model for predicting and should return a dataframe with the same structure. The series identifier is on the index. Defaults to None.	`None`
`after_predict_callback`	`callable`	Function to call on the predictions before updating the targets. This function will take a pandas Series with the predictions and should return another one with the same structure. The series identifier is on the index. Defaults to None.	`None`
`input_size`	`int`	Maximum training samples per serie in each window. If None, will use an expanding window. Defaults to None.	`None`
`weight_col`	`str`	Column that contains the sample weights. Defaults to None.	`None`

Returns:

Type	Description
`dask, spark or ray DataFrame`	Predictions for each window with the series id, timestamp, target value and predictions from each model.

​

​DistributedMLForecast

​DistributedMLForecast.fit

​DistributedMLForecast.predict

​DistributedMLForecast.save

​DistributedMLForecast.load

​DistributedMLForecast.update

​DistributedMLForecast.to_local

​DistributedMLForecast.preprocess

​DistributedMLForecast.cross_validation

`DistributedMLForecast`

`DistributedMLForecast.fit`

`DistributedMLForecast.predict`

`DistributedMLForecast.save`

`DistributedMLForecast.load`

`DistributedMLForecast.update`

`DistributedMLForecast.to_local`

`DistributedMLForecast.preprocess`

`DistributedMLForecast.cross_validation`