Python Multi-threading and concurrency: Concurrent HTTP requests with threads

Last update on December 21 2024 07:47:15 (UTC/GMT +8 hours)

Write a Python program that performs concurrent HTTP requests using threads.

Sample Solution:

Python Code:

import requests
import threading
def make_request(url):
    response = requests.get(url)
    print(f"Response from {url}: {response.status_code}")
# List of URLs to make requests to
urls = [
    "https://www.example.com",
    "https://www.google.com",
    "https://www.wikipedia.org",
    "https://www.python.org"
]
# Create and start threads for each URL
threads = []
for url in urls:
    thread = threading.Thread(target=make_request, args=(url,))
    thread.start()
    threads.append(thread)
# Wait for all threads to finish
for thread in threads:
    thread.join()

Sample Output:

Response from https://www.google.com: 200
Response from https://www.wikipedia.org: 200
Response from https://www.python.org: 200
Response from https://www.example.com: 200

Explanation:

In the above exercise,

• The "make_request()" function sends an HTTP GET request to a given URL using the requests library. The function prints the URL and the response status code.
• The program creates a list of URLs you want to request. It then iterates over the URLs, creates a thread for each URL using threading.Thread, and starts the thread by calling its start method. The threads are stored in a list for later joining.
• After starting all the threads, the program uses a loop to join each thread using the join method. This ensures that the program waits for all threads to finish before proceeding.
• Finally, the program prints a message indicating that all requests have been fulfilled.

Flowchart:

Have another way to solve this solution? Contribute your code (and comments) through Disqus.

Previous: Multi-threaded quicksort implementation.