Python Large File Download with Stream Parameter example

Last update on December 21 2024 07:48:55 (UTC/GMT +8 hours)

Write a Python program to download a large file using the stream parameter and save it to the local disk.

Sample Solution:

Python Code :

import urllib3

def download_large_file(url, local_filename):
    try:
        # Create a PoolManager instance for managing HTTP connections with connection pooling
        with urllib3.PoolManager() as http:
            # Make a GET request with stream=True to download the file in chunks
            with http.request('GET', url, preload_content=False, decode_content=False) as response:
                # Check if the request was successful (status code 200)
                if response.status == 200:
                    # Open a local file for writing in binary mode
                    with open(local_filename, 'wb') as file:
                        # Download the file in chunks and save to the local disk
                        for chunk in response.stream(8192):  # Adjust chunk size as needed
                            file.write(chunk)
                    print(f"Download complete. File saved as {local_filename}")
                else:
                    print(f"Error: Unable to download file. Status Code: {response.status}")

    except urllib3.exceptions.RequestError as e:
        print(f"Network Error: {e}")
    except Exception as e:
        print(f"Error: {e}")

if __name__ == "__main__":
    # Define the URL of the large file to download
    large_file_url = 'https://example.com/large_file.zip'  # Replace with the actual URL

    # Define the local filename to save the file
    local_filename = 'downloaded_large_file.zip'

    # Download the large file and save it to the local disk
    download_large_file(large_file_url, local_filename)

Sample Output:

Error: Unable to download file. Status Code: 404

Explanation:

Here's a brief explanation of the above Python urllib3 library code:

• Import Library:
• urllib3: Used for handling HTTP requests with connection pooling.
• Define Function:
• download_large_file(url, local_filename): Downloads a large file from the specified URL and saves it to the local disk.
• Make HTTP Request with Stream:
• Use urllib3.PoolManager() to manage HTTP connections.
• Make a GET request with stream=True to download the file in chunks without loading it into memory.
• The preload_content=False and decode_content=False parameters optimize streaming.
• Check Response Status:
• Check if the response status is 200 (OK).
• If successful, open a local file for writing in binary mode.
• Download File in Chunks:
• Iterate over the response stream in chunks and write each chunk to the local file.
• Error Handling:
• Use a try-except block to catch potential "urllib3" network-related exceptions (specifically, urllib3.exceptions.RequestError).
• If a network error occurs, print an appropriate error message.
• Additionally, catch the general "Exception" block to handle unexpected errors and print an error message.
• Run the Function:
• Calls the download_large_file() function with the URL and local filename to perform the file download.

Flowchart:

Python Code Editor :

Have another way to solve this solution? Contribute your code (and comments) through Disqus.

Previous: Python HTTP Request with Network Exception Handling Example.
Next: Constructing URLs with Python urllib3: Query Parameter Encoding Explained.