Creating a session for multiple Requests with Python urllib3 PoolManager

Last update on December 21 2024 07:48:57 (UTC/GMT +8 hours)

Write a Python program that uses the urllib3 PoolManager to create and manage a session for making multiple requests.

Sample Solution:

Python Code :

# Import necessary libraries
import urllib3

# Create a custom urllib3 PoolManager to manage connections
http = urllib3.PoolManager()

# Define the base URL for the API
base_url = 'https://www.adoptapet.com/public/apis/pet_list.html'  # Replace with the actual base URL of the API

# Create a session using the PoolManager for making multiple requests
with http.request('GET', base_url, preload_content=False) as r:
    # Check if the initial request was successful (status code 200)
    if r.status == 200:
        print("Initial request successful.")
        # Extract and print the response content
        print(r.data.decode('utf-8'))

        # Make additional requests within the same session
        for endpoint in ['/resource1', '/resource2', '/resource3']:
            url = base_url + endpoint
            # Make a GET request to each endpoint
            response = http.request('GET', url)
            
            # Check if the request was successful (status code 200)
            if response.status == 200:
                print(f"Request to {url} successful:")
                print(response.data.decode('utf-8'))
            else:
                print(f"Error: Unable to fetch data from {url}. Status Code: {response.status}")
    else:
        print(f"Error: Unable to establish a connection. Status Code: {r.status}")

# Close the urllib3 connection pool
http.clear()

Sample Output:

Error: Unable to establish a connection. Status Code: 401

Explanation:

Here's a brief explanation of the above Python urllib3 library code:

• urllib3.PoolManager() is used to create a custom connection pool manager.
• base_url is the API base URL. Replace it with the actual base URL of the API you are working with.
• The initial request is made to the base URL to establish the session using http.request('GET', base_url, preload_content=False).
• If the initial request succeeds, additional requests to specific endpoints are made within the same session.
• The session helps reuse the underlying connection for multiple requests, improving efficiency.
• The http.clear() method closes the "urllib3" connection pool.

Flowchart:

Python Code Editor :

Have another way to solve this solution? Contribute your code (and comments) through Disqus.

Previous: Making API Requests with digest authentication in Python urllib3.
Next: Implementing a Custom SSL Context for Secure HTTPS requests in Python urllib3.