get list of pandas dataframe columns based on data type

Question

If I have a dataframe with the following columns    1  NAME                                     object 2  On Time                                      object 3  On Budget                                    object 4   actual hr                                  float64 5  Baseline Start Date                  datetime64 ns  6  Forecast Start Date                  datetime64 ns     I would like to be able to say  here is a dataframe  give me a list of the columns which are of type Object or of type DateTime   I have a function which converts numbers  Float64  to two decimal places  and I would like to use this list of dataframe columns  of a particular type  and run it through this function to convert them all to 2dp   Maybe   For c in col list  if c dtype    Something  list   List append c

User · Answer

You can use boolean mask on the dtypes attribute:

In [11]: df = pd.DataFrame([[1, 2.3456, 'c']])

In [12]: df.dtypes
Out[12]: 
0      int64
1    float64
2     object
dtype: object

In [13]: msk = df.dtypes == np.float64  # or object, etc.

In [14]: msk
Out[14]: 
0    False
1     True
2    False
dtype: bool

You can look at just those columns with the desired dtype:

In [15]: df.loc[:, msk]
Out[15]: 
        1
0  2.3456

Now you can use round (or whatever) and assign it back:

In [16]: np.round(df.loc[:, msk], 2)
Out[16]: 
      1
0  2.35

In [17]: df.loc[:, msk] = np.round(df.loc[:, msk], 2)

In [18]: df
Out[18]: 
   0     1  2
0  1  2.35  c

User · Answer

If you want a list of columns of a certain type  you can use groupby    gt  gt  gt  df   pd DataFrame   1  2 3456   c    d   78    columns list  ABCDE     gt  gt  gt  df    A       B  C  D   E 0  1  2 3456  c  d  78   1 rows x 5 columns   gt  gt  gt  df dtypes A      int64 B    float64 C     object D     object E      int64 dtype  object  gt  gt  gt  g   df columns to series   groupby df dtypes  groups  gt  gt  gt  g  dtype  int64      A    E    dtype  float64      B    dtype  O      C    D     gt  gt  gt   k name  v for k  v in g items      object     C    D     int64     A    E     float64     B

User · Answer

The most direct way to get a list of columns of certain dtype e g   object    df select dtypes include  object   columns   For example    gt  gt df   pd DataFrame   1  2 3456   c    d   78    columns list  ABCDE     gt  gt df dtypes  A      int64 B    float64 C     object D     object E      int64 dtype  object   To get all  object  dtype columns    gt  gt df select dtypes include  object   columns  Index   C    D    dtype  object     For just the list    gt  gt list df select dtypes include  object   columns     C    D

User · Answer

If after 6 years you still have the issue  this should solve it    cols    c for c in df columns if df c  dtype in   object    datetime64 ns

User · Answer

list df select dtypes   object    columns    This should do the trick

User · Answer

Using dtype will give you desired column s data type   dataframe  column1   dtype     if you want to know data types of all the column at once  you can use plural of dtype as dtypes   dataframe dtypes

User · Answer

I came up with this three liner    Essentially  here s what it does     Fetch the column names and their respective data types  I am optionally outputting it to a csv      inp   pd read csv  filename csv     read input  Add read csv arguments as needed columns   pd DataFrame   column names   inp columns   datatypes   inp dtypes   columns to csv inp  columns list csv   encoding  utf-8     encoding is optional   This made my life much easier in trying to generate schemas on the fly  Hope this helps

User · Answer

use df info verbose True  where df is a pandas datafarme  by default verbose False

User · Answer

As of pandas v0 14 1  you can utilize select dtypes   to select columns by dtype In  2   df   pd DataFrame   NAME   list  abcdef         On Time    True  False    3       On Budget    False  True    3    In  3   df select dtypes include   bool    Out 3     On Budget On Time 0     False    True 1      True   False 2     False    True 3      True   False 4     False    True 5      True   False  In  4   mylist   list df select dtypes include   bool    columns   In  5   mylist Out 5     On Budget    On Time

User · Answer

for yoshiserry   def col types x pd       dtypes x dtypes     dtypes col dtypes index     dtypes type dtypes value     column types dict zip dtypes col dtypes type       return column types

User · Answer

I use infer objects       Docstring  Attempt to infer better dtypes for object columns       Attempts soft conversion of object-dtyped columns  leaving non-object   and unconvertible columns unchanged  The inference rules are the same   as during normal Series DataFrame construction    df infer objects   dtypes

User · Answer

If you want a list of only the object columns you could do   non numerics    x for x in df columns                   if not  df x  dtype    np float64                           or df x  dtype    np int64     and then if you want to get another list of only the numerics   numerics    x for x in df columns if x not in non numerics

[python] get list of pandas dataframe columns based on data type

Examples related to python

Examples related to pandas