5.3 Likelihood function

Definition 5.3 (Likelihood function) For the observed data \(x\), the function \(L(\theta|x)=f(x|\theta)\), considered as a function of \(\theta\), is called likelihood function. Mathematically \(\\\)

\[\begin{equation} L(\theta | x) = \prod_{i=1}^{n} f(x_i | \theta) \tag{5.1} \end{equation}\]

Example 5.4 Let \(X_1,\ldots,X_n\) be a sequence of (conditionally) random variables iid \(\mathcal{Ber}(\theta) \equiv \mathcal{B}(1,\theta)\). The likelihood function is given by \[\begin{equation} L(\theta|x) = \prod_{i=1}^{n} {1 \choose x_i} \theta^{x_i} (1-\theta)^{1-x_i} = \theta^{\sum_{i=1}^n x_i} (1-\theta)^{n - \sum_{i=1}^n x_i} \tag{5.2} \end{equation}\]

According to (Berger 1985, 27), the intuition behind the name ‘likelihood function’ is that the more likely the \(\theta\) is, the larger \(f(x|\theta)\).

Exercise 5.4 Whach the video Probability is not Likelihood. Find out why!!! from the channel StatQuest with Josh Starmer.


Berger, James O. 1985. Statistical Decision Theory and Bayesian Analysis. 2nd ed. Springer Science & Business Media. https://www.springer.com/gp/book/9780387960982.