replication_check#

cultivars.bayes.checks.replication_check(observed, replicated, *, kind='posterior', statistics=None, names=None, source='replications', notes=())[source]#

A predictive check on replications that came from elsewhere.

The comparison posterior_predictive_check() and prior_predictive_check() run, exposed for replications the caller simulated: from a sampler outside the package, from a result whose replication rule is their own, or from a package result whose posterior_replications output they have transformed first (a check on log returns of a model fitted to levels, say). Each discrepancy statistic is evaluated column by column on the observed panel and on every replicated panel, and the record holds those statistics; the panels themselves are discarded. The two convenience functions are this function plus the simulation and the labelling.

Parameters:

  • observed (ArrayLike) – The data, (n,) for one series or (n, k) for a panel; anything numpy.asarray() accepts. A one-dimensional series is promoted to a single column.

  • replicated (ArrayLike) – The replicated data sets, (R, n) or (R, n, k) to match, every replication the same shape as observed. At least 20 are required, since fewer cannot resolve a tail probability to better than \(0.05\).

  • kind (str) – "posterior" or "prior". It changes no number, only how summary() tells the reader to interpret the tail probabilities.

  • statistics (Sequence[str] | None) – Discrepancy statistics to compute, from mean, sd, min, max, skewness, kurtosis (excess), acf1 (lag-one autocorrelation), and arch1 (lag-one autocorrelation of the squared demeaned series), in the order the rows should carry them; duplicates are dropped. None selects mean, sd, acf1, kurtosis, arch1, which between them detect a misplaced level, a wrong scale, missed persistence, fat tails, and volatility clustering.

  • names (Sequence[str] | None) – One label per column of observed. Defaults to y1 ... yk, or y for a single series.

  • source (str) – What was checked, for the summary title.

  • notes (Sequence[str]) – Remarks to carry on the record, prepended to the summary’s notes; the place to state a caveat about how the replications were made.

Return type:

PredictiveCheckTest

Returns:

The PredictiveCheckTest, with observed of shape (m, k) and replicated of shape (R, m, k) for m statistics.

Raises:

  • SpecificationError – If kind is neither "posterior" nor "prior", if a statistic name is unknown or the request is empty, or if names does not have one label per column.

  • DimensionError – If observed is not one- or two-dimensional, if replicated is not two- or three-dimensional or its replications are not the shape of observed, or if there are fewer than 20 replications.

  • NumericalError – If any replication contains a non-finite value.

Note

A statistic that is undefined on some replication – acf1 of a constant series – is nan there and gives that cell a nan tail probability, which is reported but never flagged. Finite but pathological replications (an explosive draw that reaches \(10^{300}\)) are accepted; they show up as a replicated quantile band that dwarfs the observed statistic, which for a prior check is exactly the finding.

See also

References

Gelman, A., Meng, X.-L., & Stern, H. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statistica Sinica, 6(4), 733-760.

Example

Replications from a model the caller wrote – here an AR(1) whose coefficient is drawn from a posterior-like distribution – checked against the series:

>>> import numpy as np
>>> rng = np.random.default_rng(0)
>>> y = rng.standard_normal(100)
>>> rep = rng.standard_normal((100, 100))
>>> check = replication_check(y, rep, statistics=["mean", "sd"])
>>> check.observed.shape, check.pvalues.shape
((2, 1), (2, 1))
>>> check.names, check.source
(('y',), 'replications')

Too few replications are refused, since the resulting p-values would be meaningless:

>>> replication_check(y, rep[:10])
Traceback (most recent call last):
    ...
cultivars.exceptions.DimensionError: 10 replications; at least 20 are needed ...
On this page