Topic 17018 min readStructured Tutorial

Process Creation in Python

Quick Answer

Process Creation explains process creation is a fundamental concept in programming that allows a program to run multiple tasks simultaneously.

Learning Objectives

Explain the purpose of Process Creation in a practical learning context.
Identify the main ideas, terms, and decisions involved in Process Creation.
Apply Process Creation in a simple real-world scenario or practice task.

Introduction

Process creation is a fundamental concept in programming that allows a program to run multiple tasks simultaneously.

In Python, you can create new processes to improve performance and handle concurrent operations efficiently.

Concurrency is not parallelism, but parallelism is concurrency.

Understanding Process Creation

A process is an instance of a program in execution. Creating a new process means starting a separate program flow that runs independently.

Python provides several ways to create and manage processes, mainly through the multiprocessing and subprocess modules.

Processes run independently and have separate memory space.
Process creation enables parallel execution of code.
Useful for CPU-bound tasks and running external programs.

Multiprocessing Module

The multiprocessing module allows you to create processes that run Python functions concurrently.

It provides a Process class to spawn new processes easily.

Supports process-based parallelism.
Offers synchronization primitives like Locks and Queues.
Works similarly to threading but uses separate memory.

Subprocess Module

The subprocess module lets you spawn new processes, connect to their input/output/error pipes, and obtain their return codes.

It is commonly used to run external commands or programs from within Python.

Runs external programs and commands.
Provides functions like subprocess.run(), subprocess.Popen().
Allows capturing output and error streams.

Creating Processes with Multiprocessing

To create a process using the multiprocessing module, you define a function and pass it to a Process object.

Then you start the process using the start() method and wait for it to finish with join().

Running External Commands with Subprocess

The subprocess module is ideal for running shell commands or other programs from Python.

You can run a command and wait for it to complete, capturing its output if needed.

Practical Example

This example creates a new process that runs the worker function concurrently with the main process.

This example runs the echo command using subprocess and captures its output.

Examples

Multiprocessing Example: Creating a New Process

import multiprocessing

def worker():
    print('Worker process is running')

if __name__ == '__main__':
    p = multiprocessing.Process(target=worker)
    p.start()
    p.join()
    print('Main process finished')

This example creates a new process that runs the worker function concurrently with the main process.

Subprocess Example: Running a Shell Command

import subprocess

result = subprocess.run(['echo', 'Hello from subprocess'], capture_output=True, text=True)
print('Output:', result.stdout.strip())

This example runs the echo command using subprocess and captures its output.

Best Practices

Use multiprocessing for CPU-bound tasks to leverage multiple cores.
Use subprocess to run external programs or shell commands safely.
Always handle exceptions when working with processes to avoid orphan processes.
Use join() to ensure child processes complete before the main program exits.
Avoid sharing mutable state between processes; use queues or pipes for communication.

Common Mistakes

Confusing threads with processes; processes have separate memory spaces.
Not calling join() on processes, which can lead to premature program exit.
Using subprocess without proper input/output handling, causing deadlocks.
Modifying global variables expecting changes to reflect across processes.
Ignoring platform differences in process creation behavior.

Hands-on Exercise

Create a Multiprocessing Program

Write a Python program that creates two processes. Each process should print a different message and then exit.

Expected output: Two messages printed from separate processes.

Hint: Use multiprocessing.Process and define separate target functions for each process.

Run a Shell Command with Subprocess

Write a Python script that uses subprocess to run the 'ls' or 'dir' command and prints the output.

Expected output: List of files and directories printed to the console.

Hint: Use subprocess.run() with capture_output=True and text=True parameters.

Interview Questions

What is the difference between threading and multiprocessing in Python?

Interview

Threading runs multiple threads within the same process sharing memory, suitable for I/O-bound tasks. Multiprocessing runs separate processes with independent memory, ideal for CPU-bound tasks.

How do you create a new process in Python?

Interview

You can create a new process using the multiprocessing.Process class by passing a target function and then starting it with start().

When should you use the subprocess module?

Interview

Use subprocess when you need to run external commands or programs from Python and interact with their input/output streams.