I have an array of floats (some normal numbers, some nans) that is coming out of an apply on a pandas dataframe.
For some reason, numpy.isnan is failing on this array, however as shown below, each element is a float, numpy.isnan runs correctly on each element, the type of the variable is definitely a numpy array.
What's going on?!
set([type(x) for x in tester])
Out[59]: {float}
tester
Out[60]:
array([-0.7000000000000001, nan, nan, nan, nan, nan, nan, nan, nan, nan,
nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan,
nan, nan], dtype=object)
set([type(x) for x in tester])
Out[61]: {float}
np.isnan(tester)
Traceback (most recent call last):
File "<ipython-input-62-e3638605b43c>", line 1, in <module>
np.isnan(tester)
TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
set([np.isnan(x) for x in tester])
Out[65]: {False, True}
type(tester)
Out[66]: numpy.ndarray
A great substitute for np.isnan() and pd.isnull() is
for i in range(0,a.shape[0]):
if(a[i]!=a[i]):
//do something here
//a[i] is nan
since only nan is not equal to itself.
On top of @unutbu answer, you could coerce pandas numpy object array to native (float64) type, something along the line
import pandas as pd
pd.to_numeric(df['tester'], errors='coerce')
Specify errors='coerce' to force strings that can't be parsed to a numeric value to become NaN. Column type would be dtype: float64
, and then isnan
check should work
Make sure you import csv file using Pandas
import pandas as pd
condition = pd.isnull(data[i][j])
Source: Stackoverflow.com