Manually Implementing the Chain Rule

Construct a composite function, manually compute local gradients layer by layer and multiply them, then verify with sympy.

After completing this case, you will understand:The core of backpropagation is the chain rule—each layer only computes its own local gradient, and the final gradient is the product of the gradients of each layer.

Real-Life Introduction

The Domino Production Line

A factory has three processes: raw material → semi-finished A → semi-finished B → finished product. You want to know: "If the raw material is increased by 1 kg, how many more finished products will there be?"

You don't need to derive the formula for the entire process from scratch. Each process only calculates its own "local ratio": if semi-finished A increases by 1, how many more semi-finished B will there be? If semi-finished B increases by 1, how many more finished products? Multiply the three ratios and you get the answer.

This isthe chain rule—each layer only cares about itself, and the final impact = the product of the impacts of each layer.

Intuitive Understanding

Construct a three-layer composite function:

x
u = g(x)
x^2+1
v = f(u)
3*sin(u)
y = h(v)
v^2
\[ \frac{dy}{dx} = \underbrace{\frac{dy}{dv}}_{h'(v)} \cdot \underbrace{\frac{dv}{du}}_{f'(u)} \cdot \underbrace{\frac{du}{dx}}_{g'(x)} \]

Hands-On Python Practice

Example

import sympy as sp
import numpy as np

def g(x):    return x ** 2 + 1
def f(u):    return 3 * np.sin(u)
def g_prime(x): return 2 * x
def f_prime(u): return 3 * np.cos(u)

x0 = 1.5

# ---- Manual Chain Rule ----
u0 = g(x0)
dy_du = f_prime(u0)    # Layer 2 local gradient
du_dx = g_prime(x0)    # Layer 1 local gradient
manual_grad = dy_du * du_dx

print("=" * 55)
print("EXAMPLE Manual Chain Rule")
print(f"x0 = {x0}, u = g(x0) = {u0:.4f}")
print(f"dy/du = 3*cos(u) = {dy_du:.4f}")
print(f"du/dx = 2*x0     = {du_dx:.4f}")
print(f"Chain multiplication: dy/dx = {manual_grad:.4f}")

# ---- sympy automatic differentiation (standard answer) ----
x = sp.Symbol('x')
y_expr = 3 * sp.sin(x**2 + 1)
auto_grad = float(sp.diff(y_expr, x).subs(x, x0))
print(f"\n"EXAMPLE sympy automatic differentiation = {auto_grad:.4f}")
print(f"The two methods are consistent: {np.isclose(manual_grad, auto_grad)}")

# ---- Extension: three-layer composite ----
def h(v):     return v ** 2
def h_prime(v): return 2 * v
v0 = f(u0)
dy_dv = h_prime(v0)
grad_3layer = dy_dv * dy_du * du_dx

print(f"\n"EXAMPLE three-layer gradient propagation: ")
print(f" Layer 3: {dy_dv:.4f} x Layer 2: {dy_du:.4f} x Layer 1: {du_dx:.4f} = {grad_3layer:.4f}")
=======================================================
EXAMPLE 手动链式法则
x0 = 1.5, u = g(x0) = 3.2500
dy/du = 3*cos(u) = -2.9838
du/dx = 2*x0     = 3.0000
链式相乘: dy/dx = -8.9514

EXAMPLE sympy 自动求导 = -8.9514
两种方法一致: True

EXAMPLE 三层梯度传递：
  第3层: 6.5000 x 第2层: -2.9838 x 第1层: 3.0000 = -58.1841

Application Scenarios in AI

ScenarioRelationship with the Chain Rule
BackpropagationEach layer of a neural network only computes its own local gradient, then passes it to the previous layer.
PyTorch autogradAutomatically track every operation to build a computation graph, and automatically apply the chain rule during backpropagation
Gradient checkingAfter manually implementing backpropagation, verify with numerical gradients—this is Method 1 vs Method 2 in this case
Other extensions