cultivars.univariate.mean_function#
Learned conditional means: AR-NN and TAR-NN.
Every other family in this package writes the conditional mean as a formula in a handful of named coefficients. These two do not. The mean is a function learned from the lagged levels by a plug-in training backend,
with the reference engine a single-hidden-layer perceptron
trained by penalized least squares and the variance concentrated out, \(\hat\sigma^2 = \text{SSR} / n\). What comes back is a callable, not a parameter vector – so these results have no coefficient table at all, and the entire summary is diagnostics.
That absence is the design, not a gap. The network’s weights are
individually meaningless: permuting the hidden units, or flipping the
sign of a unit’s input weights together with its output weight, leaves
\(f\) identical. Printing them would invite interpretation of
numbers that carry none. What the results expose instead is the fitted
function itself through predict, plus the quantities that actually
tell you whether the fit is any good – fit share, capacity relative to
sample size, and the engine that produced it.
The two members differ in how the sample is partitioned.
ARNN learns one function of the \(p\) lagged levels over
the whole sample, so the nonlinearity is entirely inside the learner.
TARNN splits on an observed threshold variable
\(z_{t-d}\) at a fixed \(r\) and learns a separate function
per regime, so the nonlinearity is partly explicit – an abrupt change
of function at the split – and partly inside each learner. The
threshold is fixed, at the median by default, rather than searched:
with a nonlinear learner per regime, a grid search would retrain both
networks at every candidate split.
Two commitments shape the surface. First, the training backend is a
plug-in. Anything exposing fit(features, target) -> MeanPredictor
satisfies the MeanFunctionEngine protocol, so a torch, jax, or
scikit-learn learner drops in by adapting one method. The package ships
one reference implementation and maintains no others; the result
records which engine trained it, because with a plug-in backend the
engine is part of the specification and a summary that omitted it
would not identify the model. Second, three things are reported with
more care than usual, all consequences of the learner being nonlinear
and regularized. Information criteria penalize a count of free
parameters, but a weight-decayed network does not have that many
effective degrees of freedom – the penalty shrinks the effective
dimension below the nominal weight count, by an amount that depends on
the penalty and the data – so the counts are reported, are useful for
choosing hidden units within one engine configuration, and are not
comparable to a linear model’s. A likelihood-ratio test against a
linear null is refused: under linearity the input-to-hidden weights are
unidentified and the hidden-to-output weights sit on the boundary at
zero, so the statistic is neither chi-squared nor pivotal, the same
obstacle the threshold family hits by a different route. And the
reported log-likelihood is a concentrated Gaussian one at a penalized
fit, a goodness-of-fit summary rather than a maximized likelihood;
r_squared is the headline number.
Layout. ARNN and TARNN validate order through
validate_order and the engine on _MeanFunctionModel in
_internals; TARNN adds delay, threshold, trim
and an aligned threshold_variable on _NeuralThresholdModel.
_fit_family builds the features with lag_matrix from _core,
calls engine.fit once (or once per regime, after trailing_lag
produces the split variable), and packs a _NeuralAutoRegressionFit
or _NeuralThresholdFit. NumpyMLPEngine in
_internals._engines is the reference backend, returning a
MeanPredictor from _internals._predictors. The results extend
_MeanFunctionResult, whose _capacity_note, _criteria_note
and _engine_note are the shared diagnostics, and each emits its
mean as a NonlinearSSM through
NonlinearSSM._lag_stack_state_space. Parametric nonlinear means
with coefficients are threshold and
smooth_transition.
References
Kuan, C.-M., & White, H. (1994). Artificial neural networks: An econometric perspective. Econometric Reviews, 13(1), 1-91.
Terasvirta, T., Lin, C.-F., & Granger, C. W. J. (1993). Power of the neural network linearity test. Journal of Time Series Analysis, 14(2), 209-220.
Trapletti, A., Leisch, F., & Hornik, K. (2000). Stationary and integrated autoregressive neural network processes. Neural Computation, 12(10), 2427-2450.
Moody, J. (1992). The effective number of parameters: An analysis of generalization and regularization in nonlinear learning systems. In Advances in Neural Information Processing Systems 4, 847-854.
Example
Capacity chosen by BIC within one engine, and why the same BIC must not be read against the linear fit:
>>> import numpy as np
>>> from cultivars.univariate.autoregression import AR
>>> from cultivars.engine._internals._engines import NumpyMLPEngine
>>> rng = np.random.default_rng(0)
>>> y = np.zeros(600)
>>> for t in range(1, 600):
... y[t] = 0.8 * np.tanh(2.0 * y[t - 1]) + 0.3 * rng.standard_normal()
>>> fits = [ARNN(y, order=1, engine=NumpyMLPEngine(hidden_units=h)).fit() for h in (2, 8, 16)]
>>> [fit.n_learner_parameters for fit in fits]
[7, 25, 49]
>>> fits[0].compare(*fits[1:], criterion="bic").rows[0][2]
'8'
>>> linear = AR(y, order=1).fit()
>>> bool(fits[0].ssr < np.sum(linear.resid**2))
True
>>> bool(fits[1].information_criteria.bic > linear.information_criteria.bic)
True
Classes#
Neural autoregression: one learned function of |
|
Neural threshold autoregression: one learned function per regime. |
|
A fitted neural autoregression. |
|
The plug-in training backend for a nonlinear mean function. |
|
A fitted two-regime neural threshold autoregression. |