(sampler_stats)=
:::{post} May 31, 2022 :tags: diagnostics :category: beginner :author: Meenal Jhajharia, Christian Luhmann :::
When checking for convergence or when debugging a badly behaving sampler, it is often helpful to take a closer look at what the sampler is doing. For this purpose some samplers export statistics for each generated sample.
As a minimal example we sample from a standard normal distribution:
Note: NUTS provides the following statistics (these are internal statistics that the sampler uses, you don't need to do anything with them when using PyMC, to learn more about them, {class}pymc.NUTS.The sample statistics variables are defined as follows:
process_time_diff: The time it took to draw the sample, as defined by the python standard library time.process_time. This counts all the CPU time, including worker processes in BLAS and OpenMP.
step_size: The current integration step size.
diverging: (boolean) Indicates the presence of leapfrog transitions with large energy deviation from starting and subsequent termination of the trajectory. “large” is defined as max_energy_error going over a threshold.
lp: The joint log posterior density for the model (up to an additive constant).
energy: The value of the Hamiltonian energy for the accepted proposal (up to an additive constant).
energy_error: The difference in the Hamiltonian energy between the initial point and the accepted proposal.
perf_counter_diff: The time it took to draw the sample, as defined by the python standard library time.perf_counter (wall time).
perf_counter_start: The value of time.perf_counter at the beginning of the computation of the draw.
n_steps: The number of leapfrog steps computed. It is related to tree_depth with n_steps <= 2^tree_dept.
max_energy_error: The maximum absolute difference in Hamiltonian energy between the initial point and all possible samples in the proposed tree.
acceptance_rate: The average acceptance probabilities of all possible samples in the proposed tree.
step_size_bar: The current best known step-size. After the tuning samples, the step size is set to this value. This should converge during tuning.
tree_depth: The number of tree doublings in the balanced binary tree.
Some points to Note:
InferenceData to follow {ref}ArviZ's naming convention <arviz:schema>, while some are specific to PyMC3 and keep their internal PyMC3 name in the resulting InferenceData object.InferenceData also stores additional info like the date, versions used, sampling time and tuning steps as attributes.We check if there are any divergences, if yes, how many?
In this case no divergences are found. If there are any, check this notebook for information on handling divergences.
It is often useful to compare the overall distribution of the energy levels with the change of energy between successive samples. Ideally, they should be very similar:
If the overall distribution of energy levels has longer tails, the efficiency of the sampler will deteriorate quickly.
If multiple samplers are used for the same model (e.g. for continuous and discrete variables), the exported values are merged or stacked along a new axis.
Both samplers export accept, so we get one acceptance probability for each sampler:
We notice that accept sometimes takes really high values (jumps from regions of low probability to regions of much higher probability).
:::{include} ../page_footer.md :::