Pandas df.groupby() Function
groupby()It is one of the most powerful grouping operation functions in Pandas. It allows you to split data into different groups based on the values of one or more columns, and then perform various operations on each group.
In simple terms,groupby()it implements the"Split-Apply-Combine"(Split-Apply-Combine) workflow: first split the data by conditions, apply the corresponding function to each group, and finally combine the results.
This is very common in data analysis, such as calculating the average salary of employees by department, summing sales by month, and counting users by region.
Basic Syntax and Parameters
groupby()It is a member function of DataFrame, through the dot operator.to call. After calling, it returns aGroupByobject. This object does not display results directly; it needs to be used together with aggregation functions.
Syntax Format
DataFrame.groupby(by=None, axis=0, level=None, as_index=True, sort=True, group_keys=True, squeeze=False, observed=False, dropna=True)
Parameter Description
| Parameter | Type | Description | Default Value |
|---|---|---|---|
| by | str, list, or dict | The column name or list of column names used for grouping. If it is a dictionary or function, grouping is based on its results. | None |
| axis | int | The axis direction for grouping; 0 means by row (default), 1 means by column. | 0 |
| level | int or str | If it is a MultiIndex, group by the specified level. | None |
| as_index | bool | If True, the grouping columns will be used as the index of the returned result; if False, the grouping columns will be kept as ordinary columns. | True |
| sort | bool | Whether to sort the group labels. Setting it to False can improve performance. | True |
| group_keys | bool | When callingapply()whether to add the grouping keys as an index in the result. |
True |
| observed | bool | If True, only the actual observed values of categorical variables are displayed, rather than all possible values. | False |
| dropna | bool | If True, groups containing NA/null values will be dropped. | True |
Return Value
- Return Type：
DataFrameGroupByorSeriesGroupByobject - Description: It returns a grouping object, not the final result. You need to call an aggregation function (such as
sum()、mean()、count()etc.) to obtain the specific calculation result.
Examples
Let's thoroughly master ... through a series of examples from simple to complexgroupby()the usage of df.groupby().
Example 1: Group by a Single Column
The most basic usage is to group by the values of a single column. Suppose we have a sales data table and need to calculate total sales by region.
Example
import pandas as pd
# Create a simple sales data DataFrame
# Simulate a table containing region, product, and sales amount
data = {
'Region': ['North', 'East', 'South', 'North', 'East', 'South', 'North', 'East'],
'Product': ['A', 'B', 'C', 'B', 'A', 'C', 'A', 'B'],
'Sales': [1000, 2000, 1500, 1800, 2200, 1600, 1200, 2100]
}
# Create DataFrame
df = pd.DataFrame(data)
print("Original data:")
print(df)
print()
# Group by the 'Region' column and calculate the total sales for each region
# as_index=True means the region is returned as the index
grouped = df.groupby('Region', as_index=True)['Sales'].sum()
print("Total sales after grouping by region:")
print(grouped)
print()
# When as_index=False, the grouping column is kept as an ordinary column
grouped_df = df.groupby('Region', as_index=False)['Sales'].sum()
print("Result when as_index=False:")
print(grouped_df)
# Create a simple sales data DataFrame
# Simulate a table containing region, product, and sales amount
data = {
'Region': ['North', 'East', 'South', 'North', 'East', 'South', 'North', 'East'],
'Product': ['A', 'B', 'C', 'B', 'A', 'C', 'A', 'B'],
'Sales': [1000, 2000, 1500, 1800, 2200, 1600, 1200, 2100]
}
# Create DataFrame
df = pd.DataFrame(data)
print("Original data:")
print(df)
print()
# Group by the 'Region' column and calculate the total sales for each region
# as_index=True means the region is returned as the index
grouped = df.groupby('Region', as_index=True)['Sales'].sum()
print("Total sales after grouping by region:")
print(grouped)
print()
# When as_index=False, the grouping column is kept as an ordinary column
grouped_df = df.groupby('Region', as_index=False)['Sales'].sum()
print("Result when as_index=False:")
print(grouped_df)
Expected output:
原始数据： 地区 产品 销售额 0 华东 B 2000 1 华南 C 1500 2 华北 A 1000 3 华东 B 1800 4 华南 C 2200 5 华北 A 1600 6 华东 A 1200 7 华南 B 2100 按地区分组后的总销售额： 地区 华东 7100 华南 7300 华北 3600 dtype: int64 as_index=False 时的结果： 地区 销售额 0 华东 7100 1 华南 7300 2 华北 3600
Code analysis:
df.groupby('地区')The data is divided into three groups according to the values in the 'Region' column: East, South, North.['销售额'].sum()This means only the 'Sales' column is aggregated by sum.as_index=True(default value) returns a Series with the region as the index;as_index=FalseWhen ..., the returned DataFrame keeps the region as an ordinary column, which is more suitable for subsequent processing.
Example 2: Group by Multiple Columns
Sometimes you need to group by multiple columns at the same time, such as grouping by region and product to calculate sales.
Example
import pandas as pd
# Create sales data
data = {
'Region': ['North', 'East', 'South', 'North', 'East', 'South', 'North', 'East'],
'Product': ['A', 'B', 'C', 'B', 'A', 'C', 'A', 'B'],
'Sales': [1000, 2000, 1500, 1800, 2200, 1600, 1200, 2100]
}
df = pd.DataFrame(data)
print("Original data:")
print(df)
print()
# Group by the 'Region' and 'Product' columns to calculate total sales
# Use a list to specify multiple grouping columns
grouped = df.groupby(['Region', 'Product'], as_index=False)['Sales'].sum()
print("Total sales after grouping by region and product:")
print(grouped)
print()
# Use pivot_table to display the results more intuitively
pivot = df.pivot_table(values='Sales', index='Region', columns='Product', aggfunc='sum', fill_value=0)
print("Displayed using pivot_table:")
print(pivot)
# Create sales data
data = {
'Region': ['North', 'East', 'South', 'North', 'East', 'South', 'North', 'East'],
'Product': ['A', 'B', 'C', 'B', 'A', 'C', 'A', 'B'],
'Sales': [1000, 2000, 1500, 1800, 2200, 1600, 1200, 2100]
}
df = pd.DataFrame(data)
print("Original data:")
print(df)
print()
# Group by the 'Region' and 'Product' columns to calculate total sales
# Use a list to specify multiple grouping columns
grouped = df.groupby(['Region', 'Product'], as_index=False)['Sales'].sum()
print("Total sales after grouping by region and product:")
print(grouped)
print()
# Use pivot_table to display the results more intuitively
pivot = df.pivot_table(values='Sales', index='Region', columns='Product', aggfunc='sum', fill_value=0)
print("Displayed using pivot_table:")
print(pivot)
Expected output:
原始数据： 地区 产品 销售额 0 华北 A 1000 1 华东 B 2000 2 华南 C 1500 3 华北 B 1800 4 华东 A 2200 5 华南 C 1600 6 华北 A 1200 7 华东 B 2100 按地区和产品分组后的总销售额： 地区 产品 销售额 0 华东 A 2200 1 华东 B 4100 2 华南 C 3100 3 华北 A 2200 4 华北 B 1800 使用 pivot_table 展示： 产品 A B C 地区 华北 2200 1800 0 华东 2200 4100 0 华南 0 0 3100
Code analysis:
['Region', 'Product']Using a list allows grouping by multiple columns at the same time, and the result produces a multi-level index.as_index=FalseWhen ..., the grouping columns are kept as ordinary columns in the result, making it convenient for viewing and subsequent processing.pivot_table()It provides a similar cross-tabulation function, presenting the grouping results in a more intuitive way.
Example 3: Grouping with Dictionaries and Functions
groupby()In addition to grouping by column names, you can also define grouping rules through dictionaries or functions, which is very useful when you need custom grouping logic.
Example
import pandas as pd
# Create student grades data
data = {
'Name': ['Zhang San', 'Li Si', 'Wang Wu', 'Zhao Liu', 'Sun Qi', 'Zhou Ba'],
'Chinese': [85, 92, 78, 88, 95, 82],
'Math': [90, 85, 92, 78, 88, 91],
'English': [88, 90, 85, 92, 87, 89]
}
df = pd.DataFrame(data)
print("Original student grades data:")
print(df)
print()
# 1. Use a dictionary for custom grouping
# Suppose we want to group by "surname" (Zhang, Li, Wang as one group, Zhao, Sun, Zhou as another group)
def get_surname_group(name):
"""Return the group name based on the surname"""
if name in ['Zhang San', 'Li Si', 'Wang Wu']:
return 'Group 1'
else:
return 'Group 2'
# Use the apply function for grouping
grouped = df.groupby(get_surname_group).mean(numeric_only=True)
print("Use a function to define custom grouping (calculate the average score of each group):")
print(grouped)
print()
# 2. Use a dictionary to map grouping for a specific column
# Divide the Chinese scores into "Excellent" and "Good" groups based on numerical intervals
score_mapping = {
'Chinese': lambda x: 'Excellent' if x >= 90 else 'Good'
}
# Group the Chinese column by condition
grouped_by_score = df.groupby(lambda x: 'Excellent' if df.loc[x, 'Chinese'] >= 90 else 'Good').mean(numeric_only=True)
print("Group by Chinese score (>=90 is Excellent):")
print(grouped_by_score)
# Create student grades data
data = {
'Name': ['Zhang San', 'Li Si', 'Wang Wu', 'Zhao Liu', 'Sun Qi', 'Zhou Ba'],
'Chinese': [85, 92, 78, 88, 95, 82],
'Math': [90, 85, 92, 78, 88, 91],
'English': [88, 90, 85, 92, 87, 89]
}
df = pd.DataFrame(data)
print("Original student grades data:")
print(df)
print()
# 1. Use a dictionary for custom grouping
# Suppose we want to group by "surname" (Zhang, Li, Wang as one group, Zhao, Sun, Zhou as another group)
def get_surname_group(name):
"""Return the group name based on the surname"""
if name in ['Zhang San', 'Li Si', 'Wang Wu']:
return 'Group 1'
else:
return 'Group 2'
# Use the apply function for grouping
grouped = df.groupby(get_surname_group).mean(numeric_only=True)
print("Use a function to define custom grouping (calculate the average score of each group):")
print(grouped)
print()
# 2. Use a dictionary to map grouping for a specific column
# Divide the Chinese scores into "Excellent" and "Good" groups based on numerical intervals
score_mapping = {
'Chinese': lambda x: 'Excellent' if x >= 90 else 'Good'
}
# Group the Chinese column by condition
grouped_by_score = df.groupby(lambda x: 'Excellent' if df.loc[x, 'Chinese'] >= 90 else 'Good').mean(numeric_only=True)
print("Group by Chinese score (>=90 is Excellent):")
print(grouped_by_score)
Expected output:
原始学生成绩数据：
姓名 语文 数学 英语
0 张三 85 90 88
1 李四 92 85 90
2 王五 78 92 85
3 赵六 88 78 92
4 孙七 95 88 87
5 周八 82 91 89
使用函数自定义分组（计算每组平均分）：
语文 数学 英语
姓名
第一组 85.000000 89.000000 87.666667
第二组 88.333333 85.666667 89.333333
按语文成绩分组（>=90为优秀）：
语文 数学 英语
语文
优秀 93.500000 86.500000 88.500000
良好 83.250000 89.500000 88.750000
Code analysis:
- Custom function grouping can handle complex grouping logic; it only needs the function to return a value for grouping.
- Using
numeric_only=Truethe parameter can restrict aggregation to numeric columns, avoiding errors when operating on non-numeric columns (such as names). - Through dictionary mapping, you can implement the requirement of "dividing different columns into different groups based on conditions."
Example 4: Iterating Over Groups After Grouping
Sometimes we need to perform more complex custom operations on each group; in this case, we can iterate over the group objects.
Example
import pandas as pd
# Create sales data
data = {
'Region': ['North', 'East', 'South', 'North', 'East', 'South'],
'Product': ['A', 'B', 'C', 'B', 'A', 'C'],
'Sales amount': [1000, 2000, 1500, 1800, 2200, 1600]
}
df = pd.DataFrame(data)
print("Raw data:")
print(df)
print()
# Iterate over each group for processing
print("Detailed information for each group:")
print("-" * 40)
for group_name, group_data in df.groupby('Region'):
print(f"nGroup name: {group_name}")
print(f"Number of rows in this group: {len(group_data)}")
print(f"Total sales for this group: {group_data['Sales amount'].sum()}")
print(f"Average sales for this group: {group_data['Sales amount'].mean():.2f}")
print("-" * 40)
# Create sales data
data = {
'Region': ['North', 'East', 'South', 'North', 'East', 'South'],
'Product': ['A', 'B', 'C', 'B', 'A', 'C'],
'Sales amount': [1000, 2000, 1500, 1800, 2200, 1600]
}
df = pd.DataFrame(data)
print("Raw data:")
print(df)
print()
# Iterate over each group for processing
print("Detailed information for each group:")
print("-" * 40)
for group_name, group_data in df.groupby('Region'):
print(f"nGroup name: {group_name}")
print(f"Number of rows in this group: {len(group_data)}")
print(f"Total sales for this group: {group_data['Sales amount'].sum()}")
print(f"Average sales for this group: {group_data['Sales amount'].mean():.2f}")
print("-" * 40)
Expected output:
原始数据： 地区 产品 销售额 0 华东 B 2000 1 华南 C 1500 2 华北 A 1000 3 华东 B 1800 4 华南 C 2200 5 华北 B 1800 每个分组的详细信息： ---------------------------------------- 分组名称: 华东 该组数据行数: 2 该组销售总额: 3800 该组平均销售额: 1900.00 ---------------------------------------- 分组名称: 华南 该组数据行数: 2 3700 1850.00 ---------------------------------------- 分组名称: 华北 该组数据行数: 2 2800 1400.00 ----------------------------------------
Code explanation:
groupby()The returned grouped object can be iterated directly; each iteration returns a tuple: (group name, data of that group).- In this way, you can perform arbitrarily complex custom operations on each group.
- In practical applications, this method is often used to generate complex reports or perform data cleaning.
Tip:
groupby()The returned GroupBy object is "lazily executed" and does not compute results immediately. Only when calling aggregation functions (such assum()、mean()、count()etc.) or iterating will the actual computation be performed.
Other extensions