M4 - Nixtla

`Other`

Other(seasonality=1, horizon=8, freq='D', name='Other', n_ts=5000, included_groups=('Weekly', 'Daily', 'Hourly'))

`Hourly`

Hourly(seasonality=24, horizon=48, freq='H', name='Hourly', n_ts=414)

`Daily`

Daily(seasonality=1, horizon=14, freq='D', name='Daily', n_ts=4227)

`Weekly`

Weekly(seasonality=1, horizon=13, freq='W', name='Weekly', n_ts=359)

`Monthly`

Monthly(seasonality=12, horizon=18, freq='M', name='Monthly', n_ts=48000)

`Quarterly`

Quarterly(seasonality=4, horizon=8, freq='Q', name='Quarterly', n_ts=24000)

`Yearly`

Yearly(seasonality=1, horizon=6, freq='Y', name='Yearly', n_ts=23000)

Download data class

`M4`

M4(source_url='https://raw.githubusercontent.com/Mcompetitions/M4-methods/master/Dataset/', naive2_forecast_url='https://github.com/Nixtla/m4-forecasts/raw/master/forecasts/submission-Naive2.zip')

`M4.async_download`

async_download(directory, group=None)

Download M4 Dataset. Parameters:

Name	Type	Description	Default
`directory`	`str`	Directory path to download dataset.	required

Example:

group = 'Hourly'
await M4.async_download('data', group=group)
df, *_ = M4.load(directory='data', group=group)
n_series = len(np.unique(df.unique_id.values))
display_str  = f'Group: {group} '
display_str += f'n_series: {n_series}'
print(display_str)

`M4.download`

download(directory, group=None)

Download M4 Dataset. Parameters:

Name	Type	Description	Default
`directory`	`str`	Directory path to download dataset.	required
`group`	`str`	Name of the group to download. If None, downloads all. Defaults to None.	`None`

`M4.load`

load(directory, group, cache=True)

Downloads and loads M4 data. Parameters:

Name	Type	Description	Default
`directory`	`str`	Directory where data will be downloaded.	required
`group`	`str`	Group name. Allowed groups: ‘Yearly’, ‘Quarterly’, ‘Monthly’, ‘Weekly’, ‘Daily’, ‘Hourly’.	required
`cache`	`bool`	If `True` saves and loads	`True`

Returns:

Type	Description
`Tuple[DataFrame, Optional[DataFrame], Optional[DataFrame]]`	Tuple[pd.DataFrame, Optional[pd.DataFrame], Optional[pd.DataFrame]]: Target time series with columns [‘unique_id’, ‘ds’, ‘y’], Static exogenous variables with columns [‘unique_id’, ‘ds’], and static variables.

Evaluation class

`M4Evaluation`

`M4Evaluation.evaluate`

evaluate(directory, group, y_hat)

Evaluates y_hat according to M4 methodology. Parameters:

Name	Type	Description	Default
`directory`	`str`	Directory where data will be downloaded.	required
`group`	`str`	Group name. Allowed groups: ‘Yearly’, ‘Quarterly’, ‘Monthly’, ‘Weekly’, ‘Daily’, ‘Hourly’.	required
`y_hat`	`Union[ndarray, str]`	Group forecasts as numpy array or benchmark url from https://github.com/Nixtla/m4-forecasts/tree/master/forecasts.	required

Returns:

Type	Description
`DataFrame`	pd.DataFrame: DataFrame with columns OWA, SMAPE, MASE and group as index.

Examples:

esrnn_url = 'https://github.com/Nixtla/m4-forecasts/raw/master/forecasts/submission-118.zip'
esrnn_evaluation = M4Evaluation.evaluate('data', 'Hourly', esrnn_url)

fforma_url = 'https://github.com/Nixtla/m4-forecasts/raw/master/forecasts/submission-245.zip'
fforma_forecasts = M4Evaluation.load_benchmark('data', 'Hourly', fforma_url)
fforma_evaluation = M4Evaluation.evaluate('data', 'Hourly', fforma_forecasts)

`M4Evaluation.load_benchmark`

load_benchmark(directory, group, source_url=None)

Downloads and loads a bechmark forecasts. Parameters:

Name	Type	Description	Default
`directory`	`str`	Directory where data will be downloaded.	required
`group`	`str`	Group name. Allowed groups: ‘Yearly’, ‘Quarterly’, ‘Monthly’, ‘Weekly’, ‘Daily’, ‘Hourly’.	required
`source_url`	`str`	Optional benchmark url obtained from https://github.com/Nixtla/m4-forecasts/tree/master/forecasts. If `None` returns Naive2.	`None`

Returns:

Type	Description
`ndarray`	np.ndarray: Numpy array of shape (n_series, horizon).

URL-based evaluation

The method evaluate from the class M4Evaluation can receive a url of a benchmark uploaded to the M4 competiton. The results compared to the on-the-fly evaluation were obtained from the official evaluation.

import numpy as np

esrnn_url = 'https://github.com/Nixtla/m4-forecasts/raw/master/forecasts/submission-118.zip'
esrnn_evaluation = M4Evaluation.evaluate('data', 'Hourly', esrnn_url)
# Test of the same evaluation as the original one
assert np.isclose(esrnn_evaluation['SMAPE'].item(), 9.328, atol=1e-3)
assert np.isclose(esrnn_evaluation['MASE'].item(), 0.893, atol=1e-3)
assert np.isclose(esrnn_evaluation['OWA'].item(), 0.440, atol=1e-3)
esrnn_evaluation

Numpy-based evaluation

Also the method evaluate can recevie a numpy array of forecasts.

import numpy as np

fforma_url = 'https://github.com/Nixtla/m4-forecasts/raw/master/forecasts/submission-245.zip'
fforma_forecasts = M4Evaluation.load_benchmark('data', 'Hourly', fforma_url)
fforma_evaluation = M4Evaluation.evaluate('data', 'Hourly', fforma_forecasts)
# Test of the same evaluation as the original one
assert np.isclose(fforma_evaluation['SMAPE'].item(), 11.506, atol=1e-3)
assert np.isclose(fforma_evaluation['MASE'].item(), 0.819, atol=1e-3)
assert np.isclose(fforma_evaluation['OWA'].item(), 0.484, atol=1e-3)
fforma_evaluation

​

​Other

​Hourly

​Daily

​Weekly

​Monthly

​Quarterly

​Yearly

​Download data class

​M4

​M4.async_download

​M4.download

​M4.load

​Evaluation class

​M4Evaluation

​M4Evaluation.evaluate

​M4Evaluation.load_benchmark

​URL-based evaluation

​Numpy-based evaluation

`Other`

`Hourly`

`Daily`

`Weekly`

`Monthly`

`Quarterly`

`Yearly`

Download data class

`M4`

`M4.async_download`

`M4.download`

`M4.load`

Evaluation class

`M4Evaluation`

`M4Evaluation.evaluate`

`M4Evaluation.load_benchmark`

URL-based evaluation

Numpy-based evaluation