Apply function to each column in a data frame observing each columns existing data type

Question

I m trying to get the min max for each column in a large data frame  as part of getting to know my data  My first try was   apply t 2 max na rm 1    It treats everything as a character vector  because the first few columns are character types  So max of some of the numeric columns is coming out as     -99 5    I then tried this   sapply t max na rm 1    but it complains about max not meaningful for factors   lapply is the same   What is confusing me is that apply thought max was perfectly meaningful for factors  e g  it returned  ZEBRA  for column 1   BTW  I took a look at Using sapply on vector of POSIXct and one of the answers says  When you use sapply  your objects are coerced to numeric       Is this what is happening to me  If so  is there an alternative apply function that does not coerce  Surely it is a common need  as one of the key features of the data frame type is that each column can be a different type

User · Answer

building on  ltamar s answer   Use summary and munge the output into something useful   library tidyr  library dplyr   df   gt      summary   gt      data frame   gt     select -Var1    gt     separate data   col Freq into   c  metric   value   sep          gt     rename column name Var2    gt     mutate value as numeric value            metric   trimws metric  both          gt       filter  is na value   - gt  metrics   It s not pretty and it is certainly not fast but it gets the job done

User · Answer

A solution using retype   from hablar to coerce factors to character or numeric type depending on feasability  I d use dplyr for applying max to each column   Code  library dplyr  library hablar     Retype   simplifies each columns type  e g  always removes factors d  lt - d   gt   retype      Check max for each column d   gt   summarise all max    Result  Not the new column types        v1 v2       v3 v4       lt dbl gt   lt chr gt   lt dbl gt   lt chr gt  1 0 974 j      1 09 J      Data    Sample data borrowed from  joran d  lt - data frame v1   runif 10   v2   letters 1 10                    v3   rnorm 10   v4   LETTERS 1 10  stringsAsFactors   TRUE

User · Answer

df  lt - head mtcars  df string  lt - c  quot a quot   quot b quot    quot c quot    quot d quot   quot e quot    quot f quot    df  my min  lt - unlist lapply df  min   my max  lt - unlist lapply df  max

User · Answer

If it were an  ordered factor  things would be different  Which is not to say I like  ordered factors   I don t  only to say that some relationships are defined for  ordered factors  that are not defined for  factors   Factors are thought of as ordinary categorical variables  You are seeing the natural sort order of factors which is alphabetical lexical order for your locale  If you want to get an automatic coercion to  numeric  for every column      dates and factors and all   then try   sapply df  function x  max as numeric x         not generally a useful result   Or if you want to test for factors first and return as you expect then   sapply  df  function x  if  factor   in  class x                   max as numeric as character x                  else   max x         Darrens comment does work better    sapply df  function x  max as character x         max does succeed with character vectors

User · Answer

If you want to learn your data summary  df  provides the min  1st quantile  median and mean  3rd quantile and max of numerical columns and the frequency of the top levels of the factor columns

User · Answer

The best way to do this is avoid base  apply functions  which coerces the entire data frame to an array  possibly losing information  If you wanted to apply a function as numeric to every column  a simple way is using mutate all from dplyr  t   gt   mutate all as numeric   Alternatively use colwise from plyr  which will  quot turn a function that operates on a vector into a function that operates column-wise on a data frame  quot  t   gt    colwise as numeric    In the special case of reading in a data table of character vectors and coercing columns into the correct data type  use type convert or type convert from readr   Less interesting answer  we can apply on each column with a for-loop  for  i in 1 nrow t     t   i   lt - parse guess t   i      I don t know of a good way of doing assignment with  apply while preserving data frame structure

User · Answer

The reason that max works with apply is that apply is coercing your data frame to a matrix first  and a matrix can only hold one data type  So you end up with a matrix of characters  sapply is just a wrapper for lapply  so it is not surprising that both yield the same error   The default behavior when you create a data frame is for categorical columns to be stored as factors  Unless you specify that it is an ordered factor  operations like max and min will be undefined  since R is assuming that you ve created an unordered factor   You can change this behavior by specifying options stringsAsFactors   FALSE   which will change the default for the entire session  or you can pass stringsAsFactors   FALSE in the data frame   construction call itself  Note that this just means that min and max will assume  alphabetical  ordering by default   Or you can manually specify an ordering for each factor  although I doubt that s what you want to do   Regardless  sapply will generally yield an atomic vector  which will entail converting everything to characters in many cases  One way around this is as follows    Some test data d  lt - data frame v1   runif 10   v2   letters 1 10                    v3   rnorm 10   v4   LETTERS 1 10  stringsAsFactors   TRUE   d 4    lt - NA   Similar function to DWin s answer           fun  lt - function x       if is numeric x   max x na rm   1       else max as character x  na rm 1          Use colwise from plyr package colwise fun  d           v1 v2       v3 v4 1 0 8478983  j 1 999435  J

[r] Apply function to each column in a data frame observing each columns existing data type

Examples related to r

Examples related to apply

Examples related to sapply