pinball_loss#

cultivars.forecast.scoring.pinball_loss(realized, quantile, tau)[source]#

Score a quantile forecast with the pinball loss, elementwise.

The proper score for a \(\tau\)-quantile forecast \(q\) of an outcome \(y\),

\[\begin{split}\ell_\tau(y, q) = \bigl(\tau - \mathbb{1}\{y < q\}\bigr)(y - q) = \begin{cases} \tau\,(y - q) & y \ge q, \\ (1 - \tau)\,(q - y) & y < q, \end{cases}\end{split}\]

which a forecaster minimizes in expectation exactly by reporting the true conditional quantile, the way the squared error is minimized by the conditional mean. Under-prediction of a high quantile is penalized at rate \(\tau\) and over-prediction at \(1 - \tau\), so the asymmetry of the level is the asymmetry of the loss. A quantile VAR’s one-step forecasts, the \(\tau\)-quantile of a predictive sample, or any rival’s quantile forecasts score the same way; BacktestResult.losses("pinball", tau=...) applies it over a backtest.

Parameters:

  • realized (ArrayLike) – Outcomes, any shape.

  • quantile (ArrayLike) – Quantile forecasts of the same shape.

  • tau (float) – The level forecast, strictly inside (0, 1).

Return type:

NDArray[double]

Returns:

The losses, same shape, negatively oriented.

Raises:

Note

Elementwise and in the variable’s units: nothing is averaged, and a (T, k) panel of forecasts scores to a (T, k) panel of losses. The loss at \(\tau = 0.5\) is half the absolute error, so a median forecast’s pinball loss and its absolute error rank forecasters identically.

See also

  • losses() – the same loss per origin over a backtest, with kind="pinball".

  • quantile() – the quantile forecasts a density backtest supplies.

  • QVAR – the quantile forecaster this scores natively.

References

Koenker, R., & Bassett, G. (1978). Regression quantiles. Econometrica, 46(1), 33-50.

Gneiting, T. (2011). Making and evaluating point forecasts. Journal of the American Statistical Association, 106(494), 746-762.

Example

Missing a 90% quantile from below costs nine times missing it from above by the same amount, and the median loss is half the absolute error:

>>> import numpy as np
>>> pinball_loss([1.0, -1.0], [0.0, 0.0], 0.9)
array([0.9, 0.1])
>>> pinball_loss([1.0, -1.0], [0.0, 0.0], 0.1)
array([0.1, 0.9])
>>> bool(np.allclose(pinball_loss([2.0, -3.0], [0.0, 0.0], 0.5), [1.0, 1.5]))
True
>>> pinball_loss([1.0], [0.0], 1.0)
Traceback (most recent call last):
    ...
cultivars.exceptions.SpecificationError: tau must lie strictly inside (0, 1); got 1.0.
On this page