Python logging Module

In programming, logging is a very important tool that helps us track program execution status, debug errors, and record important information.

Python provides a built-in logging module specifically designed to handle logging tasks. Compared to simple print statements, the logging module is more flexible and powerful, and can meet logging needs in various scenarios.

Why Use the logging Module?

Flexibility: The logging module allows you to set the log level, format, and output destination as needed.
Extensibility: You can easily output logs to different targets such as files, consoles, and networks.
Structured Logging: The logging module supports structured logging, facilitating subsequent analysis and processing.
Performance Optimization: Compared to print, the logging module is optimized for performance, making it suitable for production environments.

Basic Usage of the logging Module

1. Importing the logging Module

First, we need to import the logging module:

Example

import logging

2. Configuring Log Levels

Log levels are used to control the verbosity of logs. The logging module provides the following log levels:

DEBUG: Detailed debugging information, typically used during development.
INFO: Information about normal program execution.
WARNING: Indicates potential problems, but the program can still run normally.
ERROR: Indicates errors in the program that prevent certain functions from working properly.
CRITICAL: Indicates serious errors that may cause the program to crash.

You can set the log level with the following code:

Example

logging.basicConfig(level=logging.DEBUG)

3. Recording Logs

After setting the log level, you can use the following methods to record logs:

Example

logging.debug("This is a debug message")
logging.info("This is an info message")
logging.warning("This is a warning message")
logging.error("This is an error message")
logging.critical("This is a critical error message")

4. Log Output Format

You can customize the log output format through the basicConfig method. For example:

Example

logging.basicConfig(
    level=logging.DEBUG,
    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
    datefmt="%Y-%m-%d %H:%M:%S"
)

5. Outputting Logs to a File

By default, logs are output to the console. If you want to save logs to a file, you can configure it like this:

Example

logging.basicConfig(
    level=logging.DEBUG,
    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
    filename="app.log"
)

Advanced Usage of the logging Module

1. Using Multiple Loggers

In large projects, you may need to create independent loggers for different modules or components. You can achieve this in the following way:

Example

logger = logging.getLogger("my_logger")
logger.setLevel(logging.DEBUG)

# Create a file handler
file_handler = logging.FileHandler("my_logger.log")
file_handler.setLevel(logging.DEBUG)

# Create a console handler
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.INFO)

# Set log format
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
file_handler.setFormatter(formatter)
console_handler.setFormatter(formatter)

# Add handlers to the logger
logger.addHandler(file_handler)
logger.addHandler(console_handler)

# Record logs
logger.debug("This is a debug message")
logger.info("This is an info message")

2. Log Filters

You can use filters to control which logs need to be recorded. For example:

Example

class MyFilter(logging.Filter):
    def filter(self, record):
        return record.levelno == logging.ERROR

logger.addFilter(MyFilter())

3. Log Rotation

When log files become too large, you can use RotatingFileHandler or TimedRotatingFileHandler to implement log rotation:

Example

from logging.handlers import RotatingFileHandler

handler = RotatingFileHandler("app.log", maxBytes=1024, backupCount=3)
logger.addHandler(handler)

Common Attributes and Methods of the logging Module

1. Core Classes

Class	Description	Example
`logging.Logger`	Logger, used to emit log messages (obtained via `logging.getLogger(name)`)	`logger = logging.getLogger("my_logger")`
`logging.Handler`	Handler, determines where logs are output (e.g., file, console)	`handler = logging.FileHandler("app.log")`
`logging.Formatter`	Formatter, controls the format of log output	`formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')`
`logging.Filter`	Filter, used for more fine-grained control of log recording	`filter = logging.Filter("module.name")`

2. Common Methods of the Logger Object

Method	Description	Example
`logger.setLevel(level)`	Set the log level (e.g., `logging.DEBUG`, `logging.INFO`)	`logger.setLevel(logging.DEBUG)`
`logger.debug(msg)`	Record a DEBUG level log	`logger.debug("Debug message")`
`logger.info(msg)`	Record an INFO level log	`logger.info("Program started")`
`logger.warning(msg)`	Record a WARNING level log	`logger.warning("Low disk space")`
`logger.error(msg)`	Record an ERROR level log	`logger.error("Operation failed")`
`logger.critical(msg)`	Record a CRITICAL level log	`logger.critical("System crash")`
`logger.addHandler(handler)`	Add a handler	`logger.addHandler(handler)`
`logger.addFilter(filter)`	Add a filter	`logger.addFilter(filter)`

3. Common Handler Types

Handler Type	Description	Example
`StreamHandler`	Output to a stream (e.g., console)	`handler = logging.StreamHandler()`
`FileHandler`	Output to a file	`handler = logging.FileHandler("app.log")`
`RotatingFileHandler`	Split logs by file size	`handler = logging.RotatingFileHandler("app.log", maxBytes=1e6, backupCount=3)`
`TimedRotatingFileHandler`	Split logs by time	`handler = logging.TimedRotatingFileHandler("app.log", when="midnight")`
`SMTPHandler`	Send logs via email	`handler = logging.SMTPHandler("mail.example.com", "[email protected]", "[email protected]", "Error Log")`

4. Log Levels (Constants)

Level	Value	Description
`CRITICAL`	50	Serious error, program may not continue running
`ERROR`	40	Error, but program can still run
`WARNING`	30	Warning message (default level)
`INFO`	20	Program execution information
`DEBUG`	10	Debugging information
`NOTSET`	0	Inherit the level of the parent logger

5. Common Formatter Format Fields

Field	Description	Example Output
`%(asctime)s`	Log creation time	`2023-01-01 12:00:00,123`
`%(levelname)s`	Log level name	`INFO`
`%(message)s`	Log message content	`Program started successfully`
`%(name)s`	Logger name	`my_logger`
`%(filename)s`	File name where log was generated	`app.py`
`%(lineno)d`	Line number where log was generated	`42`
`%(funcName)s`	Function name where log was generated	`main`

6. Quick Configuration Method

Method	Description	Example
`logging.basicConfig()`	One-click configuration of log level, handler, and format (usually called at the program entry point)	`logging.basicConfig(level=logging.INFO, format='%(levelname)s - %(message)s')`

Common Parameters:

level: Set the root logger level
filename: Output to a file
filemode: File mode (e.g., 'w' for overwrite)
format: Format string
datefmt: Date format (e.g., "%Y-%m-%d %H:%M:%S")

Examples

1. Basic Configuration

Example

import logging

# Configure logging
logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    filename='app.log'
)

# Usage
logger = logging.getLogger("my_app")
logger.info("Program started")

2. Complex Configuration with Multiple Handlers

Example

import logging

# Create a logger
logger = logging.getLogger("my_module")
logger.setLevel(logging.DEBUG)

# Console handler
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.WARNING)

# File handler
file_handler = logging.FileHandler("debug.log")
file_handler.setLevel(logging.DEBUG)

# Formatting
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
console_handler.setFormatter(formatter)
file_handler.setFormatter(formatter)

# Add handlers
logger.addHandler(console_handler)
logger.addHandler(file_handler)

# Usage
logger.debug("Debug message")  # Only written to file
logger.warning("Warning!")     # Output to both console and file

3. Log Rotation

Example

from logging.handlers import RotatingFileHandler

handler = RotatingFileHandler(
    "app.log", maxBytes=1e6, backupCount=3  # 1MB per file, keep 3 backups
)
logger.addHandler(handler)