Compare DataFrame element-wise multiplication using for loop vs. * Operator

Last update on December 21 2024 07:40:51 (UTC/GMT +8 hours)

Pandas: Performance Optimization Exercise-9 with Solution

Write a Pandas program that performs element-wise multiplication on a DataFrame using a for loop vs. using the * operator. Compare the performance.

Sample Solution :

Python Code :

import pandas as pd  # Import the Pandas library
import numpy as np  # Import the NumPy library
import time  # Import the time module to measure execution time

# Create a sample DataFrame
np.random.seed(0)  # Set seed for reproducibility
data = {
    'A': np.random.randint(1, 100, size=1000000),
    'B': np.random.randint(1, 100, size=1000000)
}
df = pd.DataFrame(data)

# Perform element-wise multiplication using a for loop
start_time = time.time()  # Record the start time
result_for_loop = []
for index, row in df.iterrows():
    result_for_loop.append(row['A'] * row['B'])
result_for_loop = pd.Series(result_for_loop)
time_for_loop = time.time() - start_time  # Calculate the time taken

# Perform element-wise multiplication using the * operator
start_time = time.time()  # Record the start time
result_vectorized = df['A'] * df['B']
time_vectorized = time.time() - start_time  # Calculate the time taken

# Print the time taken for both methods
print("Time taken using for loop:", time_for_loop, "seconds")
print("Time taken using * operator:", time_vectorized, "seconds")

Output:

Time taken using for loop: 37.052802324295044 seconds
Time taken using * operator: 0.0019948482513427734 seconds

Explanation:

Import Libraries:

Import the Pandas library for data manipulation.
Import the NumPy library for generating random data.
Import the time module to measure execution time.

Create a Sample DataFrame:

Set a seed for reproducibility using np.random.seed(0).
Create a dictionary data with columns 'A' and 'B' containing random integers.
Generate a DataFrame df using the dictionary.

Perform Element-wise Multiplication Using a for loop:

Record the start time using time.time().
Initialize an empty list result_for_loop to store the multiplication results.
Iterate through each row in the DataFrame using a for loop with "df.iterrows()". Multiply the values in columns 'A' and 'B' and append the result to result_for_loop.
Convert 'result_for_loop' to a Pandas Series.
Calculate the time taken by subtracting the start time from the current time.

Perform Element-wise Multiplication Using the * Operator:

Record the start time using time.time().
Use the * operator to perform element-wise multiplication of columns 'A' and 'B'.
Store the result in 'result_vectorized'.
Calculate the time taken by subtracting the start time from the current time.

Finally display the time taken for both the for loop method and the * operator method.

Python-Pandas Code Editor:

Have another way to solve this solution? Contribute your code (and comments) through Disqus.

Previous: Optimize memory usage with Categorical data type in Pandas DataFrame.
Next: Compare performance of eval method vs. standard operations in Pandas.