All of the suggestions above work, but if you want your computations to by more efficient, you should take advantage of numpy vector operations (as pointed out here).
import pandas as pd
import numpy as np
df = pd.DataFrame ({'a' : np.random.randn(6),
'b' : ['foo', 'bar'] * 3,
'c' : np.random.randn(6)})
Example 1: looping with pandas.apply()
:
%%timeit
def my_test2(row):
return row['a'] % row['c']
df['Value'] = df.apply(my_test2, axis=1)
The slowest run took 7.49 times longer than the fastest. This could mean that an intermediate result is being cached. 1000 loops, best of 3: 481 µs per loop
Example 2: vectorize using pandas.apply()
:
%%timeit
df['a'] % df['c']
The slowest run took 458.85 times longer than the fastest. This could mean that an intermediate result is being cached. 10000 loops, best of 3: 70.9 µs per loop
Example 3: vectorize using numpy arrays:
%%timeit
df['a'].values % df['c'].values
The slowest run took 7.98 times longer than the fastest. This could mean that an intermediate result is being cached. 100000 loops, best of 3: 6.39 µs per loop
So vectorizing using numpy arrays improved the speed by almost two orders of magnitude.