Pandas Descriptive Statistics

Descriptive statistics are used to summarize and describe the basic characteristics of data. Pandas provides rich statistical functions to quickly obtain the overall distribution of data.

Basic Statistical Functions

Example

import pandas as pd
import numpy as np

# Create sample data
df = pd.DataFrame({
    "Age": [25, 30, 28, 35, 22, 40, 38, 45, 29, 31],
    "Salary": [12000, 15000, 11000, 18000, 9000, 20000, 17000, 22000, 12500, 14000],
    "Performance": [85, 92, 78, 88, 65, 95, 82, 90, 75, 89]
})

print("=== Data Overview ===")
print(f"Data shape: {df.shape}")
print(f"\nBasic statistics:")
print(df.describe())

Common Statistical Measures

Function Description Example
count() Count of non-null values df["年龄"].count()
sum() Sum df["薪资"].sum()
mean() Mean df["年龄"].mean()
median() Median df["年龄"].median()
std() Standard deviation df["绩效"].std()
var() Variance df["绩效"].var()
min() / max() Minimum/Maximum df["年龄"].min()
quantile() Quantile df["薪资"].quantile(0.25)

Calculating Quantiles

Example

import pandas as pd
import numpy as np

s = pd.Series(range(1, 101))  # 1 to 100

print("Various quantiles:")
print(f"0% (min): {s.quantile(0)}")
print(f"25%: {s.quantile(0.25)}")
print(f"50% (median): {s.quantile(0.50)}")
print(f"75%: {s.quantile(0.75)}")
print(f"100% (max): {s.quantile(1)}")
print()

# Calculate multiple quantiles at the same time
print("Batch quantiles:")
print(s.quantile([0.1, 0.2, 0.5, 0.8, 0.9]))

describe Summary

describe()Provides a comprehensive statistical summary of numeric columns in the dataset.

Example

import pandas as pd

df = pd.DataFrame({
    "Age": [25, 30, 28, 35],
    "Salary": [12000, 15000, 11000, 18000],
    "City": ["Beijing", "Shanghai", "Guangzhou", "Shenzhen"]
})

print("Numerical column statistics:")
print(df.describe())
print()

print("Include all columns:")
print(df.describe(include="all"))

Correlation Coefficient and Covariance

Example

import pandas as pd
import numpy as np

df = pd.DataFrame({
    "Age": [25, 30, 28, 35, 22],
    "Salary": [12000, 15000, 11000, 18000, 9000],
    "Performance": [85, 92, 78, 88, 65]
})

print("Correlation coefficient matrix:")
print(df.corr())
print()

# Correlation with a specific column
print("Correlation with salary:")
print(df.corr()["Salary"])

The correlation coefficient ranges from -1 to 1. The closer to 1, the stronger the positive correlation; the closer to -1, the stronger the negative correlation; close to 0 indicates no correlation.

Advanced Statistics

Example

import pandas as pd
import numpy as np

s = pd.Series([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])

# Skewness (measures the skewness of a distribution)
print(f"Skewness: {s.skew()}")

# Kurtosis (measures the peakedness of a distribution)
print(f"Kurtosis: {s.kurtosis()}")
print()

# Cumulative statistics
s = pd.Series([1, 2, 3, 4, 5])
print("Cumulative sum:")
print(s.cumsum())
print()

print("Cumulative maximum:")
print(s.cummax())
Other Extensions