Vector Addition and Scalar Multiplication -- Geometric Perspective

Vector Addition: Concatenating Two Displacements

A vector can be understood as an arrow starting from the origin — it has length and direction.

Adding two vectors means concatenating two displacements together.

Scalar Multiplication: Stretching or Shrinking a Vector

Multiplying a vector by a scalar (an ordinary number) results inChanging the vector's length while keeping its line direction unchanged。

Scalar multiplication only changes the vector's "length", not the direction of the line it lies on (negative numbers reverse the direction but it remains on the same line).

This property is calledcollinearity— the two vectors before and after scalar multiplication always lie on the same line.

Everyday Examples

Mathematical Definitions

Vector Addition

To add two vectors of the same dimension, add the corresponding elements respectively.

Vector Subtraction

Vector subtraction = adding the opposite vector: \( \mathbf{a} - \mathbf{b} = \mathbf{a} + (-\mathbf{b}) \)

Scalar Multiplication of Vectors

A scalar c multiplied by a vector means multiplying every element of the vector by c:

Quick Reference of Operation Properties

Python Hands-On

a + b = [4 3]
a - b = [2 -1]
2.5 * a = [7.5 2.5]
-a = [-3 -1]
a + 10 = [13 11]
a * 3  = [9 3]

Interactive Vector Addition Demo

Drag the endpoints of the red vector a and blue vector b below, and observe the green resultant vector a+b changing in real time. Experience the triangle law and parallelogram law:

Application Scenarios in AI

Semantic Operations of Word Vectors

The most famous discovery of word embedding models such as Word2Vec and GloVe: vector addition and subtraction have semantic meaning.

Subtraction removes the "male" feature component, and addition adds the "female" feature component. The resulting vector is closest to "queen" in the embedding space.

This kind of semantic operation is not limited to gender — "Paris - France + Italy ≈ Rome" works on the same principle. Vector addition and subtraction act as "feature editing" here.

Bias in Neural Networks

In each neuron's computation \( y = Wx + b \), +b is vector addition — the bias shifts the result of wx as a whole.

Without the bias b, the model's output at x=0 would necessarily be 0, which severely limits the model's expressive power. The bias is equivalent to giving each neuron a "default activation value".

Gradient Update

The parameter update during training \( \theta_{new} = \theta_{old} - \eta \nabla J \) is a combination of scalar multiplication (learning rate × gradient) and vector subtraction.

Scalar multiplication controls the size of each update (the larger the learning rate, the larger the step), and subtraction moves the parameters in the direction of decreasing loss.

Data Preprocessing: Mean Centering

Before training a neural network, a "mean subtraction" operation is often applied to the data: \( x_{norm} = x - \mu \). This is vector subtraction — translating the data center to the origin, which helps gradient descent converge faster.