[r] Pass a data.frame column name to a function

I'm trying to write a function to accept a data.frame (x) and a column from it. The function performs some calculations on x and later returns another data.frame. I'm stuck on the best-practices method to pass the column name to the function.

The two minimal examples fun1 and fun2 below produce the desired result, being able to perform operations on x$column, using max() as an example. However, both rely on the seemingly (at least to me) inelegant

  1. call to substitute() and possibly eval()
  2. the need to pass the column name as a character vector.

fun1 <- function(x, column){
  do.call("max", list(substitute(x[a], list(a = column))))
}

fun2 <- function(x, column){
  max(eval((substitute(x[a], list(a = column)))))
}

df <- data.frame(B = rnorm(10))
fun1(df, "B")
fun2(df, "B")

I would like to be able to call the function as fun(df, B), for example. Other options I have considered but have not tried:

  • Pass column as an integer of the column number. I think this would avoid substitute(). Ideally, the function could accept either.
  • with(x, get(column)), but, even if it works, I think this would still require substitute
  • Make use of formula() and match.call(), neither of which I have much experience with.

Subquestion: Is do.call() preferred over eval()?

This question is related to r dataframe r-faq

The answer is


With dplyr it's now also possible to access a specific column of a dataframe by simply using double curly braces {{...}} around the desired column name within the function body, e.g. for col_name:

library(tidyverse)

fun <- function(df, col_name){
   df %>% 
     filter({{col_name}} == "test_string")
} 

Personally I think that passing the column as a string is pretty ugly. I like to do something like:

get.max <- function(column,data=NULL){
    column<-eval(substitute(column),data, parent.frame())
    max(column)
}

which will yield:

> get.max(mpg,mtcars)
[1] 33.9
> get.max(c(1,2,3,4,5))
[1] 5

Notice how the specification of a data.frame is optional. you can even work with functions of your columns:

> get.max(1/mpg,mtcars)
[1] 0.09615385

This answer will cover many of the same elements as existing answers, but this issue (passing column names to functions) comes up often enough that I wanted there to be an answer that covered things a little more comprehensively.

Suppose we have a very simple data frame:

dat <- data.frame(x = 1:4,
                  y = 5:8)

and we'd like to write a function that creates a new column z that is the sum of columns x and y.

A very common stumbling block here is that a natural (but incorrect) attempt often looks like this:

foo <- function(df,col_name,col1,col2){
      df$col_name <- df$col1 + df$col2
      df
}

#Call foo() like this:    
foo(dat,z,x,y)

The problem here is that df$col1 doesn't evaluate the expression col1. It simply looks for a column in df literally called col1. This behavior is described in ?Extract under the section "Recursive (list-like) Objects".

The simplest, and most often recommended solution is simply switch from $ to [[ and pass the function arguments as strings:

new_column1 <- function(df,col_name,col1,col2){
    #Create new column col_name as sum of col1 and col2
    df[[col_name]] <- df[[col1]] + df[[col2]]
    df
}

> new_column1(dat,"z","x","y")
  x y  z
1 1 5  6
2 2 6  8
3 3 7 10
4 4 8 12

This is often considered "best practice" since it is the method that is hardest to screw up. Passing the column names as strings is about as unambiguous as you can get.

The following two options are more advanced. Many popular packages make use of these kinds of techniques, but using them well requires more care and skill, as they can introduce subtle complexities and unanticipated points of failure. This section of Hadley's Advanced R book is an excellent reference for some of these issues.

If you really want to save the user from typing all those quotes, one option might be to convert bare, unquoted column names to strings using deparse(substitute()):

new_column2 <- function(df,col_name,col1,col2){
    col_name <- deparse(substitute(col_name))
    col1 <- deparse(substitute(col1))
    col2 <- deparse(substitute(col2))

    df[[col_name]] <- df[[col1]] + df[[col2]]
    df
}

> new_column2(dat,z,x,y)
  x y  z
1 1 5  6
2 2 6  8
3 3 7 10
4 4 8 12

This is, frankly, a bit silly probably, since we're really doing the same thing as in new_column1, just with a bunch of extra work to convert bare names to strings.

Finally, if we want to get really fancy, we might decide that rather than passing in the names of two columns to add, we'd like to be more flexible and allow for other combinations of two variables. In that case we'd likely resort to using eval() on an expression involving the two columns:

new_column3 <- function(df,col_name,expr){
    col_name <- deparse(substitute(col_name))
    df[[col_name]] <- eval(substitute(expr),df,parent.frame())
    df
}

Just for fun, I'm still using deparse(substitute()) for the name of the new column. Here, all of the following will work:

> new_column3(dat,z,x+y)
  x y  z
1 1 5  6
2 2 6  8
3 3 7 10
4 4 8 12
> new_column3(dat,z,x-y)
  x y  z
1 1 5 -4
2 2 6 -4
3 3 7 -4
4 4 8 -4
> new_column3(dat,z,x*y)
  x y  z
1 1 5  5
2 2 6 12
3 3 7 21
4 4 8 32

So the short answer is basically: pass data.frame column names as strings and use [[ to select single columns. Only start delving into eval, substitute, etc. if you really know what you're doing.


Another way is to use tidy evaluation approach. It is pretty straightforward to pass columns of a data frame either as strings or bare column names. See more about tidyeval here.

library(rlang)
library(tidyverse)

set.seed(123)
df <- data.frame(B = rnorm(10), D = rnorm(10))

Use column names as strings

fun3 <- function(x, ...) {
  # capture strings and create variables
  dots <- ensyms(...)
  # unquote to evaluate inside dplyr verbs
  summarise_at(x, vars(!!!dots), list(~ max(., na.rm = TRUE)))
}

fun3(df, "B")
#>          B
#> 1 1.715065

fun3(df, "B", "D")
#>          B        D
#> 1 1.715065 1.786913

Use bare column names

fun4 <- function(x, ...) {
  # capture expressions and create quosures
  dots <- enquos(...)
  # unquote to evaluate inside dplyr verbs
  summarise_at(x, vars(!!!dots), list(~ max(., na.rm = TRUE)))
}

fun4(df, B)
#>          B
#> 1 1.715065

fun4(df, B, D)
#>          B        D
#> 1 1.715065 1.786913
#>

Created on 2019-03-01 by the reprex package (v0.2.1.9000)


As an extra thought, if is needed to pass the column name unquoted to the custom function, perhaps match.call() could be useful as well in this case, as an alternative to deparse(substitute()):

df <- data.frame(A = 1:10, B = 2:11)

fun <- function(x, column){
  arg <- match.call()
  max(x[[arg$column]])
}

fun(df, A)
#> [1] 10

fun(df, B)
#> [1] 11

If there is a typo in the column name, then would be safer to stop with an error:

fun <- function(x, column) max(x[[match.call()$column]])
fun(df, typo)
#> Warning in max(x[[match.call()$column]]): no non-missing arguments to max;
#> returning -Inf
#> [1] -Inf

# Stop with error in case of typo
fun <- function(x, column){
  arg <- match.call()
  if (is.null(x[[arg$column]])) stop("Wrong column name")
  max(x[[arg$column]])
}

fun(df, typo)
#> Error in fun(df, typo): Wrong column name
fun(df, A)
#> [1] 10

Created on 2019-01-11 by the reprex package (v0.2.1)

I do not think I would use this approach since there is extra typing and complexity than just passing the quoted column name as pointed in the above answers, but well, is an approach.


If you are trying to build this function within an R package or simply want to reduce complexity, you can do the following:

test_func <- function(df, column) {
  if (column %in% colnames(df)) {
    return(max(df[, column, with=FALSE])) 
  } else {
    stop(cat(column, "not in data.frame columns."))
  }
}

The argument with=FALSE "disables the ability to refer to columns as if they are variables, thereby restoring the “data.frame mode” (per CRAN documentation). The if statement is a quick way to catch if the column name provided is within the data.frame. Could also use tryCatch error handling here.


Examples related to r

How to get AIC from Conway–Maxwell-Poisson regression via COM-poisson package in R? R : how to simply repeat a command? session not created: This version of ChromeDriver only supports Chrome version 74 error with ChromeDriver Chrome using Selenium How to show code but hide output in RMarkdown? remove kernel on jupyter notebook Function to calculate R2 (R-squared) in R Center Plot title in ggplot2 R ggplot2: stat_count() must not be used with a y aesthetic error in Bar graph R multiple conditions in if statement What does "The following object is masked from 'package:xxx'" mean?

Examples related to dataframe

Trying to merge 2 dataframes but get ValueError How to show all of columns name on pandas dataframe? Python Pandas - Find difference between two data frames Pandas get the most frequent values of a column Display all dataframe columns in a Jupyter Python Notebook How to convert column with string type to int form in pyspark data frame? Display/Print one column from a DataFrame of Series in Pandas Binning column with python pandas Selection with .loc in python Set value to an entire column of a pandas dataframe

Examples related to r-faq

What does "The following object is masked from 'package:xxx'" mean? What does "Error: object '<myvariable>' not found" mean? How do I deal with special characters like \^$.?*|+()[{ in my regex? What does %>% function mean in R? How to plot a function curve in R Use dynamic variable names in `dplyr` Error: unexpected symbol/input/string constant/numeric constant/SPECIAL in my code How should I deal with "package 'xxx' is not available (for R version x.y.z)" warning? How to select the row with the maximum value in each group R data formats: RData, Rda, Rds etc