# [python] Counting the number of non-NaN elements in a numpy ndarray in Python

I need to calculate the number of non-NaN elements in a numpy ndarray matrix. How would one efficiently do this in Python? Here is my simple code for achieving this:

``````import numpy as np

def numberOfNonNans(data):
count = 0
for i in data:
if not np.isnan(i):
count += 1
return count
``````

Is there a built-in function for this in numpy? Efficiency is important because I'm doing Big Data analysis.

Thnx for any help!

This question is related to `python` `numpy` `matrix` `nan`

``````np.count_nonzero(~np.isnan(data))
``````

`~` inverts the boolean matrix returned from `np.isnan`.

`np.count_nonzero` counts values that is not 0\false. `.sum` should give the same result. But maybe more clearly to use `count_nonzero`

Testing speed:

``````In [23]: data = np.random.random((10000,10000))

In [24]: data[[np.random.random_integers(0,10000, 100)],:][:, [np.random.random_integers(0,99, 100)]] = np.nan

In [25]: %timeit data.size - np.count_nonzero(np.isnan(data))
1 loops, best of 3: 309 ms per loop

In [26]: %timeit np.count_nonzero(~np.isnan(data))
1 loops, best of 3: 345 ms per loop

In [27]: %timeit data.size - np.isnan(data).sum()
1 loops, best of 3: 339 ms per loop
``````

`data.size - np.count_nonzero(np.isnan(data))` seems to barely be the fastest here. other data might give different relative speed results.

An alternative, but a bit slower alternative is to do it over indexing.

``````np.isnan(data)[np.isnan(data) == False].size

In [30]: %timeit np.isnan(data)[np.isnan(data) == False].size
1 loops, best of 3: 498 ms per loop
``````

The double use of `np.isnan(data)` and the `==` operator might be a bit overkill and so I posted the answer only for completeness.

#### Quick-to-write alterantive

Even though is not the fastest choice, if performance is not an issue you can use:

`sum(~np.isnan(data))`.

#### Performance:

``````In [7]: %timeit data.size - np.count_nonzero(np.isnan(data))
10 loops, best of 3: 67.5 ms per loop

In [8]: %timeit sum(~np.isnan(data))
10 loops, best of 3: 154 ms per loop

In [9]: %timeit np.sum(~np.isnan(data))
10 loops, best of 3: 140 ms per loop
``````

To determine if the array is sparse, it may help to get a proportion of nan values

``````np.isnan(ndarr).sum() / ndarr.size
``````

If that proportion exceeds a threshold, then use a sparse array, e.g. - https://sparse.pydata.org/en/latest/