Topic 22642 min readStructured Tutorial

Python Logging Tutorial

Introduction to Python Logging

Logging is an essential part of software development that helps track events during program execution.

Python provides a built-in logging module that allows developers to record messages with different severity levels.

This tutorial will guide you through the basics of Python logging, including setup, usage, and best practices.

Logging is the lifeline of debugging and monitoring in production systems.

What is Logging?

Logging is the process of recording information about a program's execution to help developers understand its behavior.

It is useful for debugging, monitoring, and auditing applications in development and production environments.

Records runtime events and errors
Helps diagnose issues without stopping the program
Supports different severity levels like DEBUG, INFO, WARNING, ERROR, and CRITICAL

Python's Logging Module Overview

Python's logging module provides a flexible framework for emitting log messages from Python programs.

It supports multiple output destinations, message formatting, and severity filtering.

Built-in and easy to use
Configurable via code or configuration files
Supports hierarchical loggers and handlers

Logging Levels in Python
Level	Numeric Value	Description
DEBUG	10	Detailed information, typically of interest only when diagnosing problems.
INFO	20	Confirmation that things are working as expected.
WARNING	30	An indication that something unexpected happened, or indicative of some problem in the near future.
ERROR	40	Due to a more serious problem, the software has not been able to perform some function.

Basic Logging Setup

To start logging in Python, you can use the basicConfig() function to configure the logging system.

This function sets the log level, output format, and destination.

Call logging.basicConfig() once at the start of your program
Set the log level to control which messages are recorded
Specify a format string to customize log message appearance

Logging Example

Here is a simple example demonstrating how to use Python's logging module.

Advanced Logging Concepts

Beyond basic logging, Python supports advanced features like multiple handlers, filters, and custom formatters.

These features allow you to send logs to different destinations and format them differently.

Handlers direct logs to files, consoles, or remote servers
Filters allow selective logging based on criteria
Formatters customize the log message layout

Handlers

Handlers determine where the log messages go. Common handlers include StreamHandler for console output and FileHandler for writing to files.

StreamHandler: outputs logs to console or streams
FileHandler: writes logs to a file
RotatingFileHandler: manages log file size by rotating files
SMTPHandler: sends logs via email

Formatters

Formatters define the layout of log messages, including timestamps, log levels, and message content.

Use format strings with placeholders like %(asctime)s, %(levelname)s, %(message)s
Customize formats to include module name, line number, or other context

Best Practices for Logging in Python

Following best practices ensures your logs are useful and maintainable.

Use appropriate log levels to categorize messages
Avoid logging sensitive information
Use descriptive messages to aid debugging
Configure logging once at application startup
Use rotating file handlers to manage log file size
Test logging configuration in development before production

Common Mistakes to Avoid

Be aware of common pitfalls when using logging.

Not configuring logging properly, resulting in no output
Logging too much at DEBUG level in production, causing performance issues
Using print statements instead of logging
Ignoring exceptions in logging handlers
Logging sensitive data like passwords or personal information

Examples

Basic Python Logging Example

import logging

logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')

logging.debug('This is a debug message')
logging.info('This is an info message')
logging.warning('This is a warning message')
logging.error('This is an error message')
logging.critical('This is a critical message')

This example sets up basic logging with INFO level and a custom format, then logs messages at various severity levels.

Best Practices

Set the logging level appropriately for development and production environments.
Use descriptive and clear log messages.
Avoid logging sensitive or personal data.
Use rotating file handlers to prevent log files from growing indefinitely.
Centralize logging configuration to avoid duplication.
Test logging outputs regularly to ensure logs are captured as expected.

Common Mistakes

Using print statements instead of logging for important messages.
Not configuring logging, resulting in missing logs.
Logging sensitive information such as passwords or tokens.
Setting the log level too low in production, causing performance degradation.
Ignoring exceptions raised by logging handlers.

Hands-on Exercise

Create a Logger with File Output

Write a Python script that logs messages of different levels to a file named 'app.log' with timestamps.

Expected output: A file 'app.log' containing timestamped log messages at various levels.

Hint: Use logging.FileHandler and set an appropriate formatter.

Implement Rotating Log Files

Modify your logging setup to use RotatingFileHandler to limit log file size to 1MB and keep 3 backups.

Expected output: Log files rotate automatically when size exceeds 1MB, keeping up to 3 backup files.

Hint: Use logging.handlers.RotatingFileHandler and configure maxBytes and backupCount.

Interview Questions

What are the different logging levels in Python and when should you use them?

Interview

Python logging levels include DEBUG for detailed diagnostic information, INFO for general events, WARNING for unexpected situations, ERROR for serious problems, and CRITICAL for severe errors that may cause program termination.

How do you configure logging to write messages to a file in Python?

Interview

You can configure logging to write to a file by creating a FileHandler and adding it to the logger, or by using logging.basicConfig with the filename parameter.

Summary

Python's logging module is a powerful tool to record runtime information and diagnose issues.

Understanding logging levels, handlers, and formatters helps you create effective logging strategies.

Following best practices and avoiding common mistakes ensures your logs are useful and secure.

FAQ

Why should I use logging instead of print statements?

Logging provides different severity levels, flexible output destinations, and better control over message formatting, making it more suitable for production applications than print statements.

Can I disable logging messages in production?

Yes, by setting the logging level to WARNING or higher, you can suppress DEBUG and INFO messages in production environments.

How do I include timestamps in my log messages?

Use a formatter with the '%(asctime)s' placeholder in the logging format string to include timestamps.