PyTorch torch.nn.HuberLoss Function

PyTorch torch.nn 参考手册PyTorch torch.nn Reference Manual

torch.nn.HuberLossIt is the Huber loss function in PyTorch.

It is a combination of MSE and MAE, which is robust to outliers and has stable gradients.

Function Definition

torch.nn.HuberLoss(delta=1.0, reduction='mean')

Parameters:

  • delta: the switch point; beyond this value, MAE is used
  • reduction: aggregation method

Mathematical Principle

L(y, ŷ) = 0.5 * (y - ŷ)², if |y - ŷ| ≤ δ
L(y, ŷ) = δ * |y - ŷ| - 0.5 * δ², if |y - ŷ| > δ

Usage Examples

Example 1: Basic Usage

Example

import torch
import torch.nn as nn

criterion = nn.HuberLoss(delta=1.0)

pred = torch.tensor([1.0, 2.0, 3.0, 10.0])
target = torch.tensor([1.5, 2.5, 3.5, 8.0])

loss = criterion(pred, target)
print("Huber Loss:", loss.item())

# Compare with MSE
mse = nn.MSELoss()(pred, target)
print("MSE Loss:", mse.item())

Example 2: Regression Training

Example

import torch
import torch.nn as nn

model = nn.Linear(10, 1)
criterion = nn.HuberLoss()
optimizer = torch.optim.Adam(model.parameters())

# Training
X = torch.randn(100, 10)
y = torch.randn(100, 1)

model.train()
optimizer.zero_grad()
output = model(X)
loss = criterion(output, y)
loss.backward()
optimizer.step()

print(Training loss:, loss.item())

Use Cases

  • Regression tasks: data with outliers
  • Object detection: bounding box regression
  • Robust training: noisy data

PyTorch torch.nn 参考手册PyTorch torch.nn Reference Manual

Other Extensions