-
Notifications
You must be signed in to change notification settings - Fork 47
Multivariate gaussian mixture model
Brian Lau edited this page May 17, 2017
·
2 revisions
This is a Matlab version of a multivariate gaussian mixture model by Dr. Maggie Lieu.
Generate a mixture of three gaussians with four dimensions:
rng(1234)
% First cluster
mu1 = [0 0 0 0];
sigma1 = eye(4)*0.1;
norm1 = mvnrnd(mu1,sigma1,30);
% Second cluster
mu2 = [7 7 7 7];
sigma2 = sigma1;
norm2 = mvnrnd(mu2,sigma2,30);
% Third cluster
mu3 = [3 3 3 3];
sigma3 = sigma1;
norm3 = mvnrnd(mu3,sigma3,30);
norms = [norm1 ; norm2 ; norm3];
[N,Dim] = size(norms);
mixture_data = struct('N',N,'D',Dim,'K',3,'y',norms);
Model definition:
mixture_model = {
'data {'
' int D; // number of dimensions'
' int K; // number of gaussians'
' int N; // number of data'
' vector[D] y[N]; // data'
'}'
'parameters {'
' simplex[K] theta; // mixing proportions'
' ordered[D] mu[K]; // mixture component means'
' cholesky_factor_corr[D] L[K]; // cholesky factor of covariance'
'}'
'model {'
' real ps[K];'
' for(k in 1:K){'
' mu[k] ~ normal(0,3);'
' L[k] ~ lkj_corr_cholesky(4);'
' }'
' for (n in 1:N){'
' for (k in 1:K){'
' // increment log probability of the gaussian'
' ps[k] = log(theta[k]) + multi_normal_cholesky_lpdf(y[n] | mu[k], L[k]);'
' }'
' target += log_sum_exp(ps);'
' }'
'}'
};
fit = stan('model_code',mixture_model,'data',mixture_data,...
'iter',1000,'warmup',1000,'chains',1);
print(fit);
Inference for Stan model: anon_model_model
1 chains: each with iter=(1000); warmup=(0); thin=(1); 1000 iterations saved.
Warmup took (33) seconds, 33 seconds total
Sampling took (31) seconds, 31 seconds total
Mean MCSE StdDev 5% 50% 95% N_Eff N_Eff/s R_hat
lp__ -4.4e+02 3.7e-01 5.1e+00 -4.5e+02 -4.4e+02 -4.4e+02 189 6.0 1.0e+00
accept_stat__ 9.2e-01 4.0e-03 1.1e-01 7.1e-01 9.5e-01 1.0e+00 812 26 1.0e+00
stepsize__ 7.4e-02 7.1e-16 5.0e-16 7.4e-02 7.4e-02 7.4e-02 0.50 0.016 1.0e+00
treedepth__ 5.4e+00 1.8e-02 5.0e-01 5.0e+00 5.0e+00 6.0e+00 784 25 1.0e+00
n_leapfrog__ 5.2e+01 6.0e-01 1.7e+01 3.1e+01 6.3e+01 6.3e+01 808 26 1.0e+00
divergent__ 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
energy__ 4.6e+02 4.6e-01 6.5e+00 4.5e+02 4.6e+02 4.7e+02 200 6.4 1.0e+00
theta[1] 3.4e-01 1.7e-03 4.9e-02 2.6e-01 3.4e-01 4.2e-01 865 28 1.0e+00
theta[2] 3.3e-01 1.7e-03 5.2e-02 2.5e-01 3.3e-01 4.1e-01 925 30 1.0e+00
theta[3] 3.3e-01 1.7e-03 4.9e-02 2.5e-01 3.3e-01 4.2e-01 868 28 1.0e+00
mu[1,1] 2.9e+00 7.0e-03 1.4e-01 2.6e+00 2.9e+00 3.1e+00 399 13 1.0e+00
mu[1,2] 2.9e+00 7.3e-03 1.3e-01 2.7e+00 2.9e+00 3.1e+00 340 11 1.0e+00
mu[1,3] 3.0e+00 8.8e-03 1.4e-01 2.7e+00 3.0e+00 3.2e+00 253 8.1 1.0e+00
mu[1,4] 3.2e+00 2.0e-02 1.6e-01 2.9e+00 3.2e+00 3.4e+00 63 2.0 1.0e+00
mu[2,1] 6.8e+00 1.5e-02 1.3e-01 6.6e+00 6.8e+00 7.0e+00 79 2.5 1.0e+00
mu[2,2] 6.9e+00 9.7e-03 1.1e-01 6.8e+00 7.0e+00 7.1e+00 119 3.8 1.0e+00
mu[2,3] 7.1e+00 7.9e-03 1.0e-01 7.0e+00 7.1e+00 7.3e+00 170 5.4 1.0e+00
mu[2,4] 7.2e+00 4.1e-03 1.0e-01 7.0e+00 7.2e+00 7.4e+00 640 20 1.0e+00
mu[3,1] -9.3e-02 7.4e-03 9.9e-02 -2.8e-01 -7.6e-02 3.8e-02 179 5.7 1.0e+00
mu[3,2] 3.8e-02 1.1e-02 1.1e-01 -1.2e-01 3.1e-02 2.2e-01 92 2.9 1.0e+00
mu[3,3] 1.3e-01 1.2e-02 1.0e-01 -1.2e-02 1.2e-01 3.3e-01 75 2.4 1.0e+00
mu[3,4] 2.2e-01 1.6e-02 1.3e-01 4.2e-02 2.0e-01 4.4e-01 65 2.1 1.0e+00
L[1,1,1] 1.0e+00 2.1e-17 6.7e-16 1.0e+00 1.0e+00 1.0e+00 1000 32 1.0e+00
L[1,1,2] 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
L[1,1,3] 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
L[1,1,4] 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
L[1,2,1] 6.3e-01 1.4e-01 5.0e-01 -8.3e-01 8.1e-01 9.0e-01 13 0.41 1.1e+00
L[1,2,2] 5.8e-01 4.1e-03 1.1e-01 4.3e-01 5.7e-01 7.9e-01 693 22 1.0e+00
L[1,2,3] 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
L[1,2,4] 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
L[1,3,1] 4.8e-01 1.2e-01 5.9e-01 -7.9e-01 7.6e-01 8.7e-01 25 0.79 1.0e+00
L[1,3,2] 3.0e-01 1.7e-02 2.0e-01 -4.2e-02 3.1e-01 6.0e-01 140 4.5 1.0e+00
L[1,3,3] 5.4e-01 6.3e-03 1.0e-01 4.0e-01 5.2e-01 7.3e-01 262 8.4 1.0e+00
L[1,3,4] 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
L[1,4,1] 6.6e-03 1.7e-01 7.2e-01 -8.2e-01 -3.0e-01 8.8e-01 17 0.54 1.0e+00
L[1,4,2] -2.5e-01 4.1e-02 2.6e-01 -6.5e-01 -2.7e-01 1.6e-01 39 1.3 1.0e+00
L[1,4,3] 1.0e-01 1.1e-02 1.9e-01 -1.9e-01 9.9e-02 4.2e-01 284 9.1 1.0e+00
L[1,4,4] 5.5e-01 4.1e-03 1.0e-01 4.1e-01 5.3e-01 7.5e-01 667 21 1.0e+00
L[2,1,1] 1.0e+00 2.1e-17 6.7e-16 1.0e+00 1.0e+00 1.0e+00 1000 32 1.0e+00
L[2,1,2] 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
L[2,1,3] 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
L[2,1,4] 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
L[2,2,1] 4.5e-01 1.7e-01 6.8e-01 -8.7e-01 8.0e-01 9.0e-01 16 0.51 1.0e+00
L[2,2,2] 5.7e-01 3.5e-03 1.1e-01 4.1e-01 5.6e-01 7.8e-01 1000 32 1.0e+00
L[2,2,3] 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
L[2,2,4] 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
L[2,3,1] -6.9e-01 4.8e-02 4.1e-01 -9.0e-01 -8.2e-01 6.8e-01 72 2.3 1.0e+00
L[2,3,2] -2.2e-01 4.5e-02 2.7e-01 -5.6e-01 -2.8e-01 3.1e-01 35 1.1 1.0e+00
L[2,3,3] 4.8e-01 4.2e-03 9.4e-02 3.6e-01 4.7e-01 6.6e-01 494 16 1.0e+00
L[2,3,4] 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
L[2,4,1] -4.5e-01 1.1e-01 5.6e-01 -8.5e-01 -7.0e-01 8.0e-01 25 0.81 1.0e+00
L[2,4,2] -2.7e-01 2.2e-02 2.4e-01 -6.0e-01 -3.0e-01 2.9e-01 119 3.8 1.0e+00
L[2,4,3] 9.5e-02 9.0e-03 1.8e-01 -2.1e-01 9.4e-02 3.8e-01 397 13 1.0e+00
L[2,4,4] 5.5e-01 3.2e-03 9.8e-02 4.1e-01 5.4e-01 7.3e-01 945 30 1.0e+00
L[3,1,1] 1.0e+00 2.1e-17 6.7e-16 1.0e+00 1.0e+00 1.0e+00 1000 32 1.0e+00
L[3,1,2] 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
L[3,1,3] 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
L[3,1,4] 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
L[3,2,1] -4.3e-01 1.2e-01 6.3e-01 -8.8e-01 -7.5e-01 8.5e-01 26 0.83 1.0e+00
L[3,2,2] 6.4e-01 4.6e-03 1.2e-01 4.6e-01 6.2e-01 8.9e-01 734 23 1.0e+00
L[3,2,3] 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
L[3,2,4] 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
L[3,3,1] -7.7e-01 5.0e-02 3.6e-01 -9.3e-01 -8.7e-01 -2.0e-02 52 1.7 1.0e+00
L[3,3,2] 1.3e-01 2.9e-02 2.2e-01 -2.8e-01 1.5e-01 4.5e-01 57 1.8 1.0e+00
L[3,3,3] 4.5e-01 4.3e-03 8.5e-02 3.3e-01 4.4e-01 6.0e-01 394 13 1.0e+00
L[3,3,4] 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 0.0e+00 1000 32 nan
L[3,4,1] -8.1e-02 1.4e-01 7.3e-01 -8.4e-01 -5.4e-01 8.8e-01 28 0.89 1.1e+00
L[3,4,2] 7.4e-04 3.2e-02 2.8e-01 -4.5e-01 1.5e-02 4.6e-01 76 2.4 1.0e+00
L[3,4,3] 1.6e-01 2.8e-02 2.1e-01 -1.8e-01 1.7e-01 4.9e-01 55 1.8 1.0e+00
L[3,4,4] 5.5e-01 6.0e-03 1.1e-01 4.0e-01 5.3e-01 7.5e-01 312 10.0 1.0e+00
Samples were drawn using hmc with nuts.
For each parameter, N_Eff is a crude measure of effective sample size,
and R_hat is the potential scale reduction factor on split chains (at
convergence, R_hat=1).