Python logging module
In programming, logging is a very important tool. It helps us track the running status of programs, debug errors, and record important information.
Python provides a built-inloggingmodule specifically for handling logging tasks. Compared with simpleprintstatements,loggingthe module is more flexible and powerful, able to meet logging needs in different scenarios.
Why use the logging module?
- Flexibility：
loggingThe module allows you to set log levels, formats, and output destinations as needed. - Scalability: You can easily output logs to different targets such as files, console, network, etc.
- Structured logging：
loggingThe module supports structured logging, making it convenient for subsequent analysis and processing. - Performance optimization: Compared with
print,loggingthe module has been optimized in performance and is suitable for use in production environments.
Basic usage of the logging module
1. Import the logging module
First, we need to importloggingthe module:
Example
import logging
2. Configure the log level
The log level is used to control the verbosity of logs.loggingThe module provides the following log levels:
- DEBUG: Detailed debugging information, usually used during development.
- INFO: Information during normal program operation.
- WARNING: Indicates potential problems, but the program can still run normally.
- ERROR: Indicates errors in the program, causing some functions to not work properly.
- CRITICAL: Indicates serious errors that may cause the program to crash.
You can set the log level with the following code:
Example
logging.basicConfig(level=logging.DEBUG)
3. Record logs
After setting the log level, you can use the following methods to record logs:
Example
logging.debug("This is a debug message")
logging.info("This is a general message")
logging.warning("This is a warning message")
logging.error("This is an error message")
logging.critical("This is a critical error message")
logging.info("This is a general message")
logging.warning("This is a warning message")
logging.error("This is an error message")
logging.critical("This is a critical error message")
4. Log output format
You can usebasicConfigthe method to customize the output format of logs. For example:
Example
logging.basicConfig(
level=logging.DEBUG,
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
datefmt="%Y-%m-%d %H:%M:%S"
)
level=logging.DEBUG,
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
datefmt="%Y-%m-%d %H:%M:%S"
)
5. Output logs to a file
By default, logs are output to the console. If you want to save logs to a file, you can configure it like this:
Example
logging.basicConfig(
level=logging.DEBUG,
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
filename="app.log"
)
level=logging.DEBUG,
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
filename="app.log"
)
Advanced usage of the logging module
1. Using multiple loggers
In large projects, you may need to create independent loggers for different modules or components. This can be achieved in the following way:
Example
logger = logging.getLogger("my_logger")
logger.setLevel(logging.DEBUG)
# Create a file handler
file_handler = logging.FileHandler("my_logger.log")
file_handler.setLevel(logging.DEBUG)
# Create a console handler
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.INFO)
# Set the log format
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
file_handler.setFormatter(formatter)
console_handler.setFormatter(formatter)
# Add handlers to the logger
logger.addHandler(file_handler)
logger.addHandler(console_handler)
# Record logs
logger.debug("This is a debug message")
logger.info("This is a general message")
logger.setLevel(logging.DEBUG)
# Create a file handler
file_handler = logging.FileHandler("my_logger.log")
file_handler.setLevel(logging.DEBUG)
# Create a console handler
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.INFO)
# Set the log format
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
file_handler.setFormatter(formatter)
console_handler.setFormatter(formatter)
# Add handlers to the logger
logger.addHandler(file_handler)
logger.addHandler(console_handler)
# Record logs
logger.debug("This is a debug message")
logger.info("This is a general message")
2. Log filters
You can use filters to control which logs need to be recorded. For example:
Example
class MyFilter(logging.Filter):
def filter(self, record):
return record.levelno == logging.ERROR
logger.addFilter(MyFilter())
def filter(self, record):
return record.levelno == logging.ERROR
logger.addFilter(MyFilter())
3. Log rotation
When the log file becomes too large, you can useRotatingFileHandlerorTimedRotatingFileHandlerto implement log rotation:
Example
from logging.handlers import RotatingFileHandler
handler = RotatingFileHandler("app.log", maxBytes=1024, backupCount=3)
logger.addHandler(handler)
handler = RotatingFileHandler("app.log", maxBytes=1024, backupCount=3)
logger.addHandler(handler)
Common attributes and methods of the logging module
1. Core classes
| Class | Description | Example |
|---|---|---|
logging.Logger | Logger, used to emit log messages (obtained vialogging.getLogger(name)get) | logger = logging.getLogger("my_logger") |
logging.Handler | Handler, determines the output destination of logs (such as files, console, etc.) | handler = logging.FileHandler("app.log") |
logging.Formatter | Formatter, controls the format of log output | formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s') |
logging.Filter | Filter, used for finer control of log recording | filter = logging.Filter("module.name") |
2. Common methods of the Logger object
| Method | Description | Example |
|---|---|---|
logger.setLevel(level) | Set the log level (such aslogging.DEBUG、logging.INFO） | logger.setLevel(logging.DEBUG) |
logger.debug(msg) | Record DEBUG level logs | logger.debug("调试信息") |
logger.info(msg) | Record INFO level logs | logger.info("程序启动") |
logger.warning(msg) | Record WARNING level logs | logger.warning("磁盘空间不足") |
logger.error(msg) | Record ERROR level logs | logger.error("操作失败") |
logger.critical(msg) | Record CRITICAL level logs | logger.critical("系统崩溃") |
logger.addHandler(handler) | Add handler | logger.addHandler(handler) |
logger.addFilter(filter) | Add filter | logger.addFilter(filter) |
3. Common Handler types
| Handler type | Description | Example |
|---|---|---|
StreamHandler | Output to a stream (such as console) | handler = logging.StreamHandler() |
FileHandler | Output to a file | handler = logging.FileHandler("app.log") |
RotatingFileHandler | Split logs by file size | handler = logging.RotatingFileHandler("app.log", maxBytes=1e6, backupCount=3) |
TimedRotatingFileHandler | Split logs by time | handler = logging.TimedRotatingFileHandler("app.log", when="midnight") |
SMTPHandler | Send logs via email | handler = logging.SMTPHandler("mail.example.com", "from@example.com", "to@example.com", "Error Log") |
4. Log levels (constants)
| Level | Value | Description |
|---|---|---|
CRITICAL | 50 | Critical error, the program may not be able to continue running |
ERROR | 40 | Error, but the program can still run |
WARNING | 30 | Warning message (default level) |
INFO | 20 | Program running information |
DEBUG | 10 | Debug information |
NOTSET | 0 | Inherit the level of the parent logger |
5. Common Formatter format fields
| Field | Description | Example output |
|---|---|---|
%(asctime)s | Log creation time | 2023-01-01 12:00:00,123 |
%(levelname)s | Log level name | INFO |
%(message)s | Log message content | Program started successfully. |
%(name)s | Logger name | my_logger |
%(filename)s | Filename where the log was generated | app.py |
%(lineno)d | Line number where the log was generated | 42 |
%(funcName)s | Function name where the log was generated | main |
6. Quick configuration methods
| Method | Description | Example |
|---|---|---|
logging.basicConfig() | One-click configuration of log level, handlers, and format (usually called at the program entry point) | logging.basicConfig(level=logging.INFO, format='%(levelname)s - %(message)s') |
Common parameters：
level: Set the root logger levelfilename: Output to a filefilemode: File mode (such as'w'overwrite)format: Format stringdatefmt: Date format (such as"%Y-%m-%d %H:%M:%S"）
Example
1. Basic configuration
Example
import logging
# Configure logging
logging.basicConfig(
level=logging.DEBUG,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
filename='app.log'
)
# Use
logger = logging.getLogger("my_app")
logger.info("Program started")
# Configure logging
logging.basicConfig(
level=logging.DEBUG,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
filename='app.log'
)
# Use
logger = logging.getLogger("my_app")
logger.info("Program started")
2. Complex configuration with multiple handlers
Example
import logging
# Create a logger
logger = logging.getLogger("my_module")
logger.setLevel(logging.DEBUG)
# Console handler
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.WARNING)
# File handler
file_handler = logging.FileHandler("debug.log")
file_handler.setLevel(logging.DEBUG)
# Formatting
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
console_handler.setFormatter(formatter)
file_handler.setFormatter(formatter)
# Add handlers
logger.addHandler(console_handler)
logger.addHandler(file_handler)
# Use
logger.debug("Debug message") # Write to file only
logger.warning("Warning!") # Output to both console and file
# Create a logger
logger = logging.getLogger("my_module")
logger.setLevel(logging.DEBUG)
# Console handler
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.WARNING)
# File handler
file_handler = logging.FileHandler("debug.log")
file_handler.setLevel(logging.DEBUG)
# Formatting
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
console_handler.setFormatter(formatter)
file_handler.setFormatter(formatter)
# Add handlers
logger.addHandler(console_handler)
logger.addHandler(file_handler)
# Use
logger.debug("Debug message") # Write to file only
logger.warning("Warning!") # Output to both console and file
3. Log splitting
Example
from logging.handlers import RotatingFileHandler
handler = RotatingFileHandler(
"app.log", maxBytes=1e6, backupCount=3 # 1MB per file, keep 3 backups
)
logger.addHandler(handler)