Coin Toss Simulation of Bayesian Updating
Simulate the process of repeatedly tossing a coin, and use Bayes' formula to update in real time the belief distribution of whether the coin is biased.
After completing this case study, you will understand:The core of the Bayesian method — every time new data is observed, update your belief once using the likelihood function.
Real-Life Introduction
Is the new colleague trustworthy?
On the first day, you don't know the new colleague (prior: they may be reliable or unreliable, 50% each). In the first week, they complete tasks on time every time (new evidence), and you begin to think "they are probably reliable". In the second week, they complete on time again (more evidence), and your judgment becomes more certain.
Your judgment is the posterior probability — the result of continuously revising the initial judgment with new evidence. This is the intuition of Bayesian updating.
Intuitive Understanding
We have a coin, but we don't know its probability of heads. Prior: before tossing, we consider any heads probability (0 to 1) possible. Likelihood: each time we see a result, we update using "the probability of seeing this result if the heads probability is theta". Posterior: the updated belief distribution — the more data, the more concentrated the distribution is around the true value.
Mathematical Definition
\[ P(\theta \mid \text{data}) = \frac{P(\text{data} \mid \theta) \cdot P(\theta)}{P(\text{data})} \] \[ \text{posterior}_t \propto \text{likelihood}(\text{the } t\text{-th result} \mid \theta) \times \text{posterior}_{t-1} \]Python Hands-on Practice
Example
np.random.seed(1)
true_p = 0.75 # True coin heads probability (pretend we don't know)
theta_grid = np.linspace(0, 1, 200)
# Prior: treat all theta equally
prior = np.ones_like(theta_grid)
prior /= prior.sum()
# Simulate tossing the coin 40 times
flips = np.random.binomial(1, true_p, size=40)
posterior = prior.copy()
print("EXAMPLE Bayesian updating process (true theta=0.75):\n")
for i, outcome in enumerate(flips, start=1):
likelihood = theta_grid if outcome == 1 else (1 - theta_grid)
posterior = posterior * likelihood
posterior /= posterior.sum()
if i % 10 == 0:
est = theta_grid[np.argmax(posterior)]
print(f" After {i:2d} tosses, most likely theta={est:.3f}")
# Uncertainty comparison
prior_std = np.sqrt(np.sum(theta_grid**2 * prior) - np.sum(theta_grid * prior)**2)
post_std = np.sqrt(np.sum(theta_grid**2 * posterior) - np.sum(theta_grid * posterior)**2)
print(f"\nEXAMPLE prior standard deviation: {prior_std:.4f} -> posterior standard deviation: {post_std:.4f} (reduced by a factor of {prior_std/post_std:.1f}))
EXAMPLE 贝叶斯更新过程 (真实 theta=0.75): 抛 10 次后，最可能的 theta=0.754 抛 20 次后，最可能的 theta=0.769 抛 30 次后，最可能的 theta=0.759 抛 40 次后，最可能的 theta=0.774 EXAMPLE 先验标准差: 0.2890 -> 后验标准差: 0.0642 (缩小了 4.5 倍)
Application Scenarios in AI
| Scene | The role of Bayesian methods |
|---|---|
| Naive Bayes classification | Bayes' theorem + feature independence assumption → classic classifier |
| Bayesian optimization | Use prior + observed data → posterior Gaussian process → guide hyperparameter search. |
| Variational inference | Approximating Complex Posteriors with Simple Distributions: The Mathematical Foundations of VAE |