Python Multithreading: A Step-by-Step Guide 

Welcome to a journey through the world of Python multithreading, where we simplify complex concepts and bring them to life with real-world analogies and coding examples. Let's unravel the mysteries of terms like "parallelism" and "multithreading" and address some key questions:

• What is concurrency?

• How does concurrency differ from parallelism?

• What are the distinctions between processes and threads?

• How is multithreading used in Python?

• What is the Global Interpreter Lock in Python?

Assuming you're equipped with Python's basic knowledge, let's dive deeper. If you need a refresher, our Python for finance and trading is a great starting point.

Understanding Concurrency

Concurrency in programming is akin to a chef managing multiple dishes at once. Unlike sequential programming, where tasks are handled one after another, concurrent programming juggles multiple tasks simultaneously, optimizing runtime efficiency.

Concurrency vs. Parallelism: The Differences

Concurrency is like a solo musician playing different instruments in quick succession, while parallelism resembles an orchestra where all musicians play simultaneously. In technical terms, concurrency involves managing multiple tasks on a single processor, whereas parallelism involves executing multiple tasks at the same time across multiple processors.

Processes vs. Threads: A Simplified Explanation

Picture a process as a complete workshop with its tools and space. Each workshop (process) operates independently. A thread, in this context, is akin to a worker within the workshop, sharing tools and space, thus making communication and resource sharing more efficient compared to separate workshops.

Multithreading in Python

Python's multithreading is like having a single worker (thread) in a workshop (process) at any given time, governed by the Global Interpreter Lock (GIL). The GIL ensures thread safety but limits the true simultaneous execution of threads.

Python's Multithreading Modules

Python offers the thread and threading modules for multithreading. For serious multithreading applications, the threading module is preferred due to its advanced features.

Practical Example: Multithreading in Action

Let's illustrate multithreading with an example. Consider a scenario where we have a list of integers, and we need to find both the factorial and the sum of each number. Here's how we can do it both sequentially and with multithreading.

Sequential Code Example

import time
import math

def calc_factorial(numbers):
    for n in numbers:
        print(f'Factorial of {n} = {math.factorial(n)}')
        time.sleep(0.1)

def calc_sum(numbers):
    for n in numbers:
        print(f'Sum up to {n} = {sum(range(1, n+1))}')
        time.sleep(0.1)

numbers = [3, 4, 5, 6]
start = time.time()
calc_factorial(numbers)
calc_sum(numbers)
end = time.time()
print(f'Sequential Execution Time: {end - start}')

Multithreaded Code Example

import threading

# Start threads
factorial_thread = threading.Thread(target=calc_factorial, args=(numbers,))
sum_thread = threading.Thread(target=calc_sum, args=(numbers,))

factorial_thread.start()
sum_thread.start()

# Wait for threads to complete
factorial_thread.join()
sum_thread.join()

end = time.time()
print(f'Multithreaded Execution Time: {end - start}')

What is Global Interpreter Lock (GIL) ?

It is a mechanism that prevents multiple native threads from executing Python bytecodes at once. This lock is necessary because Python's memory management is not thread-safe. To understand the GIL, let's use a real-life analogy and then look at a Python example.

Imagine a kitchen (the Python interpreter) where only one person (thread) can cook (execute) at a time, no matter how many stoves (CPUs) are available. This person must follow a recipe (Python bytecode), which involves various tasks like chopping vegetables, boiling water, etc. Even if there are multiple stoves, the kitchen rules (GIL) allow only one person in the kitchen at a time to avoid accidents (memory management issues).

If another person (another thread) wants to cook (execute tasks), they must wait outside the kitchen until the current person leaves (releases the GIL). This system ensures safety (memory integrity) but can be inefficient, as other stoves remain unused and people wait their turn.

Consider two Python threads trying to perform a computation-heavy task:

import threading
import time

def count_down(n):
    while n > 0:
        n -= 1

start_time = time.time()

# Create two threads
t1 = threading.Thread(target=count_down, args=(50000000,))
t2 = threading.Thread(target=count_down, args=(50000000,))

# Start the threads
t1.start()
t2.start()

# Wait for threads to complete
t1.join()
t2.join()

end_time = time.time()

print(f"Time taken in seconds: {end_time - start_time}")

In this example, even though there are two threads, due to the GIL, they don't truly run in parallel on multiple CPU cores. Instead, they take turns executing, resulting in a performance similar to single-threaded execution. This is like two chefs taking turns to use the one-person kitchen, thus not fully utilizing the available resources (stoves).

what is a daemon thread ?

it is a thread that runs in the background and is not essential for the program to run. It's typically used for tasks that run continuously in the background without user intervention, like background monitoring or periodic data updates. The main characteristic of a daemon thread is that the program will not wait for daemon threads to complete before exiting. When the main program terminates, daemon threads are killed automatically.

Think of a daemon thread as a security guard monitoring a building at night. The guard's job (background task) is ongoing and doesn't interfere with the primary operations of the building during the day (main program). When the building closes (the main program ends), the guard's job ends too, regardless of whether it's completed.

Python Example

Here's an example of how to use a daemon thread in Python:

import threading
import time

def background_task():
    while True:
        print("Background task is running...")
        time.sleep(1)

# Create a daemon thread
daemon_thread = threading.Thread(target=background_task)
daemon_thread.daemon = True  # Set the thread as a daemon thread

# Start the daemon thread
daemon_thread.start()

# Main program
print("Main program is running for 5 seconds.")
time.sleep(5)
print("Main program is ending.")

In this example, background_task is a simple function that continuously prints a message every second. It's run in a separate thread marked as a daemon (daemon_thread.daemon = True). The main program runs for 5 seconds and then ends. When the main program ends, the daemon thread background_task will be terminated automatically, regardless of its state.

This behavior makes daemon threads useful for tasks that you want to run in the background but don't need to complete for the program to finish.

Conclusion

In this guide, we've explored the essentials of multithreading in Python, contrasting it with concurrency and parallelism concepts. We've applied these concepts in a basic multithreaded program for calculating factorial and sum of numbers in a list. Stay tuned for more advanced multithreading techniques in Python.