moderation_analysis.ipynb

https://github.com/pymc-devs/pymc-examples/blob/main/examples/causal_inference/moderation_analysis.ipynb

(moderation_analysis)=

Bayesian moderation analysis

:::{post} March, 2022 :tags: moderation, path analysis, :category: beginner :author: Benjamin T. Vincent :::

This notebook covers Bayesian moderation analysis. This is appropriate when we believe that one predictor variable (the moderator) may influence the linear relationship between another predictor variable and an outcome. Here we look at an example where we look at the relationship between hours of training and muscle mass, where it may be that age (the moderating variable) affects this relationship._

This is not intended as a one-stop solution to a wide variety of data analysis problems, rather, it is intended as an educational exposition to show how moderation analysis works and how to conduct Bayesian parameter estimation in PyMC.

Note that this is sometimes mixed up with mediation analysis. Mediation analysis is appropriate when we believe the effect of a predictor variable upon an outcome variable is (partially, or fully) mediated through a 3rd mediating variable. Readers are referred to the textbook by {cite:t}hayes2017introduction as a comprehensive (albeit Frequentist) guide to moderation and related models as well as the PyMC example {ref}mediation_analysis._

First in the (hidden) code cell below, we define some helper functions for plotting that we will use later.

Does the effect of training upon muscularity decrease with age?

I've taken inspiration from a blog post {cite:t}vandenbergSPSS which examines whether age influences (moderates) the effect of training on muscle percentage. We might speculate that more training results in higher muscle mass, at least for younger people. But it might be the case that the relationship between training and muscle mass changes with age - perhaps training is less effective at increasing muscle mass in older age?

The schematic box and arrow notation often used to represent moderation is shown by an arrow from the moderating variable to the line between a predictor and an outcome variable.

[Image blocked: No description]

It can be useful to use consistent notation, so we will define:

$x$ as the main predictor variable. In this example it is training.
$y$ as the outcome variable. In this example it is muscle percentage.
$m$ as the moderator. In this example it is age.

The moderation model

While the visual schematic (above) is a useful shorthand to understand complex models when you already know what moderation is, you can't derive it from the diagram alone. So let us formally specify the moderation model - it defines an outcome variable $y$ as:

y \sim \mathrm{Normal}(\beta_0 + \beta_1 \cdot x + \beta_2 \cdot x \cdot m + \beta_3 \cdot m, \sigma^2)

where $y$, $x$, and $m$ are your observed data, and the following are the model parameters:

$\beta_0$ is the intercept, its value does not have that much importance in the interpretation of this model.
$\beta_1$ is the rate at which $y$ (muscle percentage) increases per unit of $x$ (training hours).
$\beta_2$ is the coefficient for the interaction term $x \cdot m$.
$\beta_3$ is the rate at which $y$ (muscle percentage) increases per unit of $m$ (age).
$\sigma$ is the standard deviation of the observation noise.

We can see that the mean $y$ is simply a multiple linear regression with an interaction term between the two predictors, $x$ and $m$.

We can get some insight into why this is the case by thinking about this as a multiple linear regression with $x$ and $m$ as predictor variables, but where the value of $m$ influences the relationship between $x$ and $y$. This is achieved by making the regression coefficient for $x$ is a function of $m$:

y \sim \mathrm{Normal}(\beta_0 + f(m) \cdot x + \beta_3 \cdot m, \sigma^2)

and if we define that as a linear function, $f(m) = \beta_1 + \beta_2 \cdot m$, we get

y \sim \mathrm{Normal}(\beta_0 + (\beta_1 + \beta_2 \cdot m) \cdot x + \beta_3 \cdot m, \sigma^2)

We can use $f(m) = \beta_1 + \beta_2 \cdot m$ later to visualise the moderation effect.

Import data

First, we will load up our example data and do some basic data visualisation. The dataset is taken from {cite:t}vandenbergSPSS but it is unclear if this corresponds to real life research data or if it was simulated.

Define the PyMC model and conduct inference

Plot the model graph to confirm it is as intended.

Visualise the trace to check for convergence.

We have good chain mixing and the posteriors for each chain look very similar, so no problems in that regard.

Visualise the important parameters

First we will use a pair plot to look at joint posterior distributions. This might help us identify any estimation issues with the interaction term (see the discussion below about multicollinearity).

And just for the sake of completeness, we can plot the posterior distributions for each of the $\beta$ parameters and use this to arrive at research conclusions.

For example, from an estimation (in contrast to a hypothesis testing) perspective, we could look at the posterior over $\beta_2$ and claim a credibly less than zero moderation effect.

Posterior predictive checks

Define a set of quantiles of $m$ that we are interested in visualising.

Visualisation in data space

Here we will plot the data alongside model posterior predictive checks. This can be a useful visual method of comparing the model predictions against the data.

Spotlight graph

We can also visualise the moderation effect by plotting $\beta_1 + \beta_2 \cdot m$ as a function of the $m$. This was named a spotlight graph, see {cite:t}spiller2013spotlights and {cite:t}mcclelland2017multicollinearity.

The expression $\beta_1 + \beta_2 \cdot \text{moderator}$ defines the rate of change of the outcome (muscle percentage) per unit of $x$ (training hours/week). We can see that as age (the moderator) increases, this effect of training hours/week on muscle percentage decreases.

Related issues: mean centering and multicollinearity

Readers should be aware that there are issues around mean-centering and multicollinearity. The original SPSS Moderation Regression Tutorial did mean-centre the predictor variables $x$ and $m$. This will have a downstream effect upon the interaction term $x \cdot m$.

One effect of mean centering is to change the interpretation of the parameter estimates. In this notebook, we did not mean center the variables which will affect the parameter estimates and their interpretation. It is not that one is correct or incorrect, but one must be cognisant of how mean-centering (or not) affects the interpretation of parameter estimates. Readers are again directed to {cite:t}hayes2017introduction for a more in-depth consideration of mean-centering in moderation analyses.

Another issue, particularly relevant to moderation analysis is multicollinearity, where one predictor variable is well-described as a linear combination of other predictors. This is clearly the case in moderation analysis as the interaction term $m \cdot x$ is by definition a linear combination of $x$ and $m$.

{cite:t}iacobucci2016mean explored the issues of mean-centering and multicollinearity and conclude:

When all is said and done, should a researcher mean center the X1 and X2 variables before computing a product term X1X2 to include in a moderated multiple regression? It depends. Mean centering is advisable when: (1) the predictor variables are measured on scales with arbitrary zeros and the researcher seeks to enhance the interpretation of the regression results vis-à-vis the variables’ means rather than the arbitrary zero points, or (2) the research questions involve testing the main effect terms in addition to the interaction term and the researcher seeks to obtain these statistical tests without the interference of the so-called nonessential multicollinearity. On the other hand, mean centering may be bypassed when: (1) the research question involves primarily the test of the interaction term, with no regard for the lower order main effect terms, or (2) the research question involves primarily the assessment of the overall fit of the model, the R2, with no interest in apportioning the explained variability across the predictors, main effects or interaction alike.

This was critiqued however by {cite:t}mcclelland2017multicollinearity who claimed that {cite:t}iacobucci2016mean made a number of errors, and that multicollinearity is a red herring:

Multicollinearity is irrelevant to the search for moderator variables, contrary to the implications of Iacobucci, Schneider, Popovich, and Bakamitsos (Behavior Research Methods, 2016, this issue). Multicollinearity is like the red herring in a mystery novel that distracts the statistical detective from the pursuit of a true moderator relationship.

They state:

Researchers using MMR [moderated multiple regression] need not compute any multicollinearity diagnostics nor worry about it at all. They need not use mean-centering or the orthogonal transformation or do anything else to avoid the purported problems of multicollinearity. The only purpose of those transformations is to facilitate understanding of MMR models.

Bearing in mind {cite:t}mcclelland2017multicollinearity took a frequentist hypothesis testing (not a Bayesian approach) their take-home points can be paraphrased as:

Fit the regression model, $y \sim \mathrm{Normal}(\beta_0 + \beta_1 \cdot x + \beta_2 \cdot x \cdot m + \beta_3 \cdot m, \sigma^2)$, with original (not mean-centred) data.
If the main interest is on the moderation effect, then focus upon $\beta_2$.
Transformations are useful if conditional relationships are to be highlighted.
"... researchers who wish to examine all possible conditional relationships or to help their readers who might want to consider other conditional relationships, should construct the [spotlight] graph..."

But readers are strongly encouraged to read {cite:t}mcclelland2017multicollinearity for more details, as well as the reply from {cite:t}iacobucci2017mean. Readers should also be aware that there are conflicting opinions and recommendations about mean centering etc in textbooks (see Further Reading below), some of which are published before 2017. None of these textbooks explicitly cite {cite:t}mcclelland2017multicollinearity, so it is unclear if the textbook authors are unaware of, agree with, or disagree with {cite:t}mcclelland2017multicollinearity.

Authors

Authored by Benjamin T. Vincent in June 2021
Updated by Benjamin T. Vincent in March 2022
Updated by Benjamin T. Vincent in February 2023 to run on PyMC v5
Updated to use az.extract by Benjamin T. Vincent in February 2023 (pymc-examples#522)

References

:::{bibliography} :filter: docname in docnames :::

Watermark

:::{include} ../page_footer.md :::