3  Intermission: cooking with Bayes

Bayesian cuisine has many concepts that some may find strange or exotic. Demistifying a handful of these concepts will allow us to prepare some delightful data analyses.

All Bayesian models need the same ingredients:

• Data \(\bf{y}\), which here include dependent and independent variables in a regression.
• A model with parameters, aka a likelihood, \(\pi(\bf{y} \mid \theta)\) that describes how the variables in our data behave and relate to each other.
• Plausible values of parameters before seeing the data, aka prior distribution, \(\pi(\theta)\). This prior distribution can condense previous empirical work or disregard absurd behaviors or relationships.

Our goal is to learn from our data to obtain a more accurate range of plausible values for \(\theta\). This new range of values—the dish we want—is called posterior distribution and we can write it as \(\pi(\theta \mid \bf{y})\). Bayes’ theorem gives us the recipe:

\[ \begin{align*} \pi(\theta \mid \bf{y}) = \frac{\pi(\theta) \times \pi(\bf{y} \mid \theta)} {\text{A fancy constant}}. \end{align*} \]

Our first impulse may be to do the math explicitly to get a specific formula. Unfortunately, the fancy constant is an ugly integral that is impossible to compute in all but the simplest models. We could also try to approximate this formula with another, simpler formula that is easier to compute. This works sometimes but is often difficult and inaccurate.

Figuring out how to actually use Bayes’ theorem is not even remotely obvious, but we can gain some intuition with a short fable. Imagine that your good friend gambling Bob gifts you his favourite pair of loaded dice, but leaves before telling you how they work. How could you find out which numbers are the good ones? A simple solution is to roll these dice a whole lot of times and record the numbers you observe each time. With enough rolls, you will get a good description of how often each possible number appears with these dice.

As with the dice, we can learn about \(\theta\) by repeatedly drawing random values to get a sample. If we follow the probabilities given by the product in Bayes’ Theorem, the random values we draw will eventually describe the plausible values for \(\theta\), i.e., its posterior distribution. With this strategy, we can ignore the fancy constant because it does not change the relative probabilities of the values we sample.

Thus, we can view Bayesian computation as rolling some metaphorical dice and then using Bayes’ theorem to transform these rolls into values that describe the plausible range of the parameters (or other quantities) we want to study.

So far we know the ingredients and the recipe to cook our dish. Now all we need is a few details on how to use the knobs and buttons on our oven (brms).