Optimize string operations in Pandas: str accessor vs. apply

Last update on April 09 2025 12:56:57 (UTC/GMT +8 hours)

15. String Operations: str Accessor vs. apply() with Custom Function

Write a Pandas program to optimize the performance of string operations on a DataFrame column by using the str accessor vs. applying a custom function with apply.

Sample Solution :

Python Code :

# Import necessary libraries
import pandas as pd
import time

# Create a sample DataFrame
num_rows = 1000000
df = pd.DataFrame({
    'text': ['example_string'] * num_rows
})

# Measure time for str accessor method
start_time = time.time()
str_accessor_result = df['text'].str.upper()
end_time = time.time()
str_accessor_time = end_time - start_time

# Define a custom function to apply
def to_upper(text):
    return text.upper()

# Measure time for apply method
start_time = time.time()
apply_result = df['text'].apply(to_upper)
end_time = time.time()
apply_time = end_time - start_time

# Print the time taken for each method
print(f"Time taken using str accessor: {str_accessor_time:.6f} seconds")
print(f"Time taken using apply method: {apply_time:.6f} seconds")

Output:

Time taken using str accessor: 0.181023 seconds
Time taken using apply method: 0.139029 seconds

Explanation:

• Import Libraries:
• Import pandas, numpy, and time.
• Create DataFrame:
• Generate a sample DataFrame with 1,000,000 rows, each containing a string.
• Time Measurement for str Accessor:
• Measure the time taken to convert strings to uppercase using the str.upper accessor.
• Define Custom Function:
• Define a custom function to convert strings to uppercase.
• Time Measurement for apply Method:
• Measure the time taken to apply the custom function using the apply method.
• Finally print the time taken for each method.

For more Practice: Solve these Related Problems:

• Write a Pandas program to perform string operations on a DataFrame column using the str accessor and measure its speed.
• Write a Pandas program to apply a custom string processing function using apply() and compare the performance with the str accessor.
• Write a Pandas program to benchmark string manipulation on a large text column using vectorized str methods versus a looped apply() function.
• Write a Pandas program to analyze the performance benefits of using the str accessor for converting text to lowercase compared to using apply().

Python-Pandas Code Editor:

Have another way to solve this solution? Contribute your code (and comments) through Disqus.

Previous: Performance comparison of cumulative Sum calculation in Pandas.
Next: Reshaping DataFrame in Pandas: pivot_table vs. manual Loop.