posterior_predictive_check#

cultivars.bayes.checks.posterior_predictive_check(result, *, statistics=None, n_replications=200, seed=None)[source]#

Replicate the sample from the posterior and compare its features with the data.

For each of n_replications retained posterior draws \(\theta^{(r)}\), the result simulates a data set \(y^{\text{rep},(r)} \sim p(y \mid \theta^{(r)})\) of the same shape as the effective sample it was fitted to, with fresh innovations and, for a conditional model, the same presample. The draws are taken evenly across the kept sequence rather than from its first stretch, so the replications span the posterior; asking for more replications than draws cycles through the draws with new innovations. The observed and replicated panels then go to replication_check(), and the result’s own remarks on how it replicates – a stochastic-volatility posterior replicating under the smoothed variance path, say – ride along as the record’s notes.

This is the check of Gelman, Meng, and Stern (1996): the same data fit the parameters and judge the fit, so the tail probabilities are conservative in the sense the PredictiveCheckTest docstring describes, and a value in a tail is a stronger finding than its size suggests.

Parameters:

  • result (ReplicatingResult) – A fitted result that can replicate its sample, that is, one satisfying ReplicatingResult by exposing observed and posterior_replications(): the Bayesian VAR family (BVAR, GibbsBVAR, StudentBVAR, BVARSV) and the stochastic-volatility posterior of SV.

  • statistics (Sequence[str] | None) – Discrepancy statistics, as for replication_check(); None selects mean, sd, acf1, kurtosis, arch1.

  • n_replications (int) – Replicated data sets, one per posterior draw used. At least 20; 200 resolves a tail probability to \(0.005\).

  • seed (int | Generator | None) – Seed or generator for the replications’ innovations. An integer gives a reproducible check; a numpy.random.Generator is used in place and advanced.

Return type:

PredictiveCheckTest

Returns:

The PredictiveCheckTest with kind="posterior", the result’s variable names, its label as source, and its replication remarks as notes.

Raises:

  • SpecificationError – If the result cannot replicate its sample – a point estimate has no posterior to replicate from, and neither has a result whose draws were not retained – or if a statistic name is unknown.

  • DimensionError – If fewer than 20 replications are asked for.

See also

References

Gelman, A., Meng, X.-L., & Stern, H. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statistica Sinica, 6(4), 733-760.

Meng, X.-L. (1994). Posterior predictive p-values. Annals of Statistics, 22(3), 1142-1160.

Example

A stochastic-volatility posterior checked on the two features that distinguish it from constant variance – the scale and the volatility clustering:

>>> import numpy as np
>>> from cultivars.univariate.stochastic_volatility import SV
>>> rng = np.random.default_rng(1)
>>> h = np.zeros(300)
>>> for t in range(1, 300):
...     h[t] = 0.95 * h[t - 1] + 0.3 * rng.standard_normal()
>>> y = np.exp(0.5 * h) * rng.standard_normal(300)
>>> post = SV(y, mean="zero").sample(n_draws=600, n_burn=200, seed=0)
>>> check = posterior_predictive_check(post, statistics=["sd", "arch1"], seed=0)
>>> check.statistics, check.names, check.kind
(('sd', 'arch1'), ('y',), 'posterior')
>>> check.n_replications, check.nobs
(200, 300)

A point estimate is refused:

>>> from cultivars.multivariate.reduced_form.vector_autoregression import VAR
>>> ols = VAR(rng.standard_normal((80, 2)), order=1).fit()
>>> posterior_predictive_check(ols)
Traceback (most recent call last):
    ...
cultivars.exceptions.SpecificationError: VARResult cannot replicate its sample: ...