GP Subpackage

Gaussian Process related subpackage.

ife_surrogate.gp.nmll_wideband(X, Y, sigma_sq, jitter, kernel)

Computes the negative marginal log-likelihood (NMLL) for a wideband (multi-output) Gaussian Process model:

\[-\log p(\mathbf{Y} | \mathbf{X}, \mathbf{\theta}, \mathbf{\sigma}^2) = -\sum_{p=1}^P \log\left( p(\mathbf{y}_p|\mathbf{X}, \mathbf{\theta}, \sigma^2_p) \right)\]

with:

\[p(\mathbf{y}_p|\mathbf{X}, \mathbf{\theta}, \sigma^2_p) = \frac{1}{2} \mathbf{y}_p^\intercal \mathbf{K}^{-1} \mathbf{y}_p + \frac{1}{2} \log\left\lvert \mathbf{K} \right\rvert + \frac{n}{2} \log(2\pi) + \frac{N}{2} \log(\sigma_p^2)\]

Parameters:

• X (Array) – Training input data of shape (N, D).

• Y (Array) – Training output data of shape (N, P), where P is the number of outputs (tasks).

• sigma_sq (Array) – Per-output noise variance, shape (P,).

• jitter (float) – Small positive constant added to the diagonal of the kernel matrix for numerical stability.

• kernel (Callable) – Kernel function used to compute covariances between inputs.

Returns:

The negative marginal log-likelihood value.

Return type:

Float

Notes

This implementation assumes independent noise across outputs (diagonal noise covariance), and scales the likelihood calculation accordingly.

ife_surrogate.gp.nlml_scalar(X, Y, jitter, kernel)

Computes the negative marginal log-likelihood (NMLL) for a scalar-output Gaussian Process model:

\[p(\mathbf{y}|\mathbf{X}, \mathbf{\theta}) = \frac{1}{2} \mathbf{y}^\intercal \mathbf{K}^{-1} \mathbf{y} + \frac{1}{2} \log\left\lvert \mathbf{K} \right\rvert + \frac{N}{2} \log(2\pi)\]

Parameters:

• X (Array) – Training input data of shape (N, D).

• Y (Array) – Training output data of shape (N, 1) or (N,).

• sigma_sq (Array) – Estimated noise variance, shape (1,) or scalar.

• jitter (float) – Small positive constant added to the diagonal of the kernel matrix for numerical stability.

• kernel (Callable) – Kernel function used to compute covariances between inputs.

Returns:

The negative marginal log-likelihood value.

Return type:

Float

Notes

This function is tailored for standard (single-output) Gaussian Process models, assuming a homoskedastic noise model.

ife_surrogate.gp.nmll_kron(X, Y, jitter, kernel)

Computes the negative marginal log-likelihood (NMLL) for a wideband (multi-output) Gaussian Process model: with K_w = diag(alpha) C_w diag(alpha) with alpha = sqrt(sigma**2)

\[-\log p(\mathbf{Y} | \mathbf{X}, \mathbf{\theta}, \mathbf{\sigma}^2) = -\sum_{p=1}^P \log\left( p(\mathbf{y}_p|\mathbf{X}, \mathbf{\theta}, \sigma^2_p) \right)\]

with:

\[\begin{split}\text{vec}(\mathbf{Y})^\top \left( \mathbf{K}_t^{-1} \otimes \mathbf{K}_x^{-1} \right) \text{vec}(\mathbf{Y}) &= \mathrm{trace} \left( \mathbf{K}_x^{-1} \, \mathbf{Y} \, \mathbf{K}_t^{-1} \, \mathbf{Y}^\top \right) \\[6pt] &= \sum_{i=1}^N \sum_{j=1}^P \left[ \mathbf{K}_x^{-1} \mathbf{Y} \right]_{ij} \; \left[ \mathbf{K}_t^{-1} (\mathbf{K}_x^{-1} \mathbf{Y})^\top \right]_{ji}\end{split}\]

Parameters:

• X (Array) – Training input data of shape (N, D).

• Y (Array) – Training output data of shape (N, P), where P is the number of outputs (tasks).

• jitter (float) – Small positive constant added to the diagonal of the kernel matrix for numerical stability.

• kernel (Callable) – Kernel function used to compute covariances between inputs.

Returns:

The negative marginal log-likelihood value.

Return type:

Float

Notes

This implementation assumes independent noise across outputs (diagonal noise covariance), and scales the likelihood calculation accordingly.

ife_surrogate.gp.predictive_mean_var_wideband(X_train, Y_train, sigma_sq, kernel, jitter, X_test)

Calculates the posterior mean and variance for a wideband (multi-output) Gaussian Process with a diagonal noise covariance:

\[\begin{split}p(\mathbf{f_{*,p}}|\mathbf{X},\mathbf{y_p},\mathbf{X_*}) =& \; \mathcal{N}(\mathbf{f_*}|\mathbf{\mu_{p}},\mathbf{\Sigma_p}) \\ \mathbf{\mu_p} =& \; \mathbf{K_*}^{T}\mathbf{K_x}^{-1} \mathbf{y_p} \\ \mathbf{\Sigma_p} =& \; \sigma_p^2 (\mathbf{K_{**}} - \mathbf{K_*}^{T}\mathbf{K_x}^{-1}\mathbf{K_*})\end{split}\]

Parameters:

• X_train (Array) – Training input data of shape (N, D).

• Y_train (Array) – Training output data of shape (N, P).

• sigma_sq (float) – Variance of the noise.

• kernel – Kernel function object used to compute covariances.

• jitter (float) – Small value added to the diagonal for numerical stability.

• X_test (Array) – Test input data of shape (M, D).

Returns:

Posterior mean and variance at the test inputs for every task.

• posterior mean: shape (M, P)

• posterior variance: shape (M,)

Return type:

Tuple[Array, Array]

ife_surrogate.gp.predictive_mean_var_scalar(X_train, Y_train, kernel, jitter, X_test)

Calculates the posterior mean and variance for a scalar output GP. Posterior distribution with posterior mean and variance:

\[\begin{split}p(\mathbf{f_{*}}|\mathbf{X},\mathbf{y},\mathbf{X_*}) =& \; \mathcal{N}(\mathbf{f_*}|\mathbf{\mu},\mathbf{\Sigma}) \\ \mathbf{\mu_p} =& \; \mathbf{K_*}^{T}\mathbf{K}^{-1} \mathbf{y} \\ \mathbf{\Sigma_p} =& \; \mathbf{K_{**}} - \mathbf{K_*}^{T}\mathbf{K}^{-1}\mathbf{K_*})\end{split}\]

Parameters:

• X_train (Array)

• Y_train (Array)

• jitter (float)

• X_test (Array)

Return type:

Tuple[Array, Array]

ife_surrogate.gp.predictive_mean_var_kron(X_train, Y_train, kernel, jitter, XW_test)

Predictive mean for Kron GP with Kw = diag(alpha) Cw diag(alpha). Alpha will be linearly interpolated for predictions. Where alpha test = func(f_test| f_train, std(Y_train))

Parameters:

• X_train (Array)

• Y_train (Array)

• jitter (float)

• XW_test (Array)

Return type:

Tuple[Array, Array]