:::{post} Sept 23, 2021 :tags: generalized linear model, hierarchical model, variational inference :category: intermediate :::
Unlike Gaussian mixture models, (hierarchical) regression models have independent variables. These variables affect the likelihood function, but are not random variables. When using mini-batch, we should take care of that.
Here, log_radon_idx_t is a dependent variable, while floor_idx_t and county_idx_t determine the independent variable.
Intercept for each county, distributed around group mean mu_a. Above we just set mu and sd to a fixed value while here we plug in a common group distribution for all a and b (which are vectors with the same length as the number of unique counties in our example).
Model prediction of radon level a[county_idx] translates to a[0, 0, 0, 1, 1, ...], we thus link multiple household measures of a county to its coefficients.
Finally, we specify the likelihood:
Random variables radon_like, associated with log_radon_idx_t, should be given to the function for ADVI to denote that as observations in the likelihood term.
On the other hand, minibatches should include the three variables above.
Then, run ADVI with mini-batch.
Check the trace of ELBO and compare the result with MCMC.
We can extract the covariance matrix from the mean field approximation and use it as the scaling matrix for the NUTS algorithm.
Also, we can generate samples (one for each chain) to use as the starting points for the sampler.
:::{include} ../page_footer.md :::