GP

GP

Referring to A Visual Exploration of Gaussian Processes for its intutive explanations.

1.Multivariate Gaussian distributions

$$\mathbf{X} = \begin{bmatrix} X_1 \\ X_2 \\ \vdots \\ X_n \end{bmatrix} \sim \mathcal{N}(\mu, \Sigma)\\ \Sigma = Cov(X_i,X_j) = E[(X_i-\mu_i)(X_j-\mu_j)^T] $$$$ P_{\mathbf X,\mathbf Y} = \begin{bmatrix} X \\ Y \end{bmatrix} \sim \mathcal N(\mu,\Sigma) = \mathcal N( \begin{bmatrix} \mu_{X} \\ \mu_{Y} \end{bmatrix},\begin{bmatrix} \Sigma_{XX} & \Sigma_{XY} \\ \Sigma_{YX} & \Sigma_{YY} \end{bmatrix} ) $$

1.1 Marginalization

$$ X \sim \mathcal{N}(\mu_{X},\Sigma_{XX}) \\ Y \sim \mathcal{N}(\mu_{Y},\Sigma_{YY}) $$

1.2 Conditioning

$$ X|Y \sim \mathcal{N}( \mu_X + \Sigma_{XY}\Sigma_{YY}^{-1}(Y - \mu_Y), \: \Sigma_{XX} - \Sigma_{XY}\Sigma_{YY}^{-1}\Sigma_{YX} ) $$$$ Y|X \sim \mathcal{N}( \mu_Y + \Sigma_{YX}\Sigma_{XX}^{-1}(X - \mu_X), \: \Sigma_{YY} - \Sigma_{YX}\Sigma_{XX}^{-1}\Sigma_{XY} ) $$