Removing nan values from an array

Question

I want to figure out how to remove nan values from my array  My array looks something like this    x    1400  1500  1600  nan  nan  nan  1700   Not in this exact configuration   How can I remove the nan values from x

User · Answer

A simplest way is    numpy nan to num x    Documentation  https   docs scipy org doc numpy reference generated numpy nan to num html

User · Answer

The accepted answer changes shape for 2d arrays  I present a solution here  using the Pandas dropna   functionality  It works for 1D and 2D arrays  In the 2D case you can choose weather to drop the row or column containing np nan   import pandas as pd import numpy as np  def dropna arr   args    kwarg       assert isinstance arr  np ndarray      dropped pd DataFrame arr  dropna  args    kwarg  values     if arr ndim  1          dropped dropped flatten       return dropped  x   np array  1400  1500  1600  np nan  np nan  np nan  1700   y   np array   1400  1500  1600    np nan  0  np nan    1700 1800 np nan       print     20   1D Case         20   nInput  n  x sep     print   ndropna  n  dropna x  sep      print   n n      20   2D Case         20   nInput  n  y sep     print   ndropna  rows   n  dropna y  sep     print   ndropna  columns   n  dropna y axis 1  sep      print   n n      20   x np logical not np isnan x    for 2D         20   nInput  n  y sep     print   ndropna  n  x np logical not np isnan x    sep       Result                        1D Case                       Input   1400  1500  1600    nan   nan   nan 1700    dropna   1400  1500  1600  1700                          2D Case                       Input    1400  1500  1600       nan    0    nan    1700  1800    nan    dropna  rows     1400  1500  1600     dropna  columns     1500        0     1800                           x np logical not np isnan x    for 2D                       Input    1400  1500  1600       nan    0    nan    1700  1800    nan    dropna   1400  1500  1600  1700

User · Answer

Try this   import math print  value for value in x if not math isnan value     For more  read on List Comprehensions

User · Answer

filter lambda v  v  v  x    works both for lists and numpy array since v  v only for NaN

User · Answer

Doing the above    x   x  numpy isnan x     or  x   x numpy logical not numpy isnan x      I found that resetting to the same variable  x  did not remove the actual nan values and had to use a different variable  Setting it to a different variable removed the nans  e g    y   x  numpy isnan x

User · Answer

jmetz s answer is probably the one most people need  however it yields a one-dimensional array  e g  making it unusable to remove entire rows or columns in matrices   To do so  one should reduce the logical array to one dimension  then index the target array  For instance  the following will remove rows which have at least one NaN value   x   x  numpy isnan x  any axis 1     See more detail here

User · Answer

If you re using numpy for your arrays  you can also use  x   x numpy logical not numpy isnan x      Equivalently   x   x  numpy isnan x      Thanks to chbrown for the added shorthand    Explanation  The inner function  numpy isnan returns a boolean logical array which has the value True everywhere that x is not-a-number  As we want the opposite  we use the logical-not operator    to get an array with Trues everywhere that x is a valid number   Lastly we use this logical array to index into the original array x  to retrieve just the non-NaN values

User · Answer

As shown by others   x  numpy isnan x     works  But it will throw an error if the numpy dtype is not a native data type  for example if it is object  In that case you can use pandas   x  pandas isna x   or x  pandas isnull x

User · Answer

If you re using numpy     first get the indices where the values are finite ii   np isfinite x     second get the values x   x ii

User · Answer

This is my approach to filter ndarray  X  for NaNs and infs      I create a map of rows without any NaN and any inf as follows   idx   np where  np isnan X   False   amp   np isinf X   False     idx is a tuple  It s second column  idx 1   contains the indices of the array  where no NaN nor inf where found across the row   Then   filtered X   X idx 1     filtered X contains X without NaN nor inf

User · Answer

For me the answer by  jmetz didn t work  however using pandas isnull   did   x   x  pd isnull x

User · Answer

Simply fill with  x   numpy array     0 99929941  0 84724713  -0 1500044     -0 79709026  numpy NaN  -0 4406645     -0 3599013  -0 63565744  -0 70251352     x numpy isnan x      555  print x        0 99929941  0 84724713 -0 1500044       -0 79709026  0 555      -0 4406645       -0 3599013  -0 63565744 -0 70251352

[python] Removing nan values from an array

Examples related to python

Examples related to arrays

Examples related to numpy

Examples related to nan