[r] Compute mean and standard deviation by group for multiple variables in a data.frame

This is an aggregation problem, not a reshaping problem as the question originally suggested -- we wish to aggregate each column into a mean and standard deviation by ID. There are many packages that handle such problems. In the base of R it can be done using aggregate like this (assuming DF is the input data frame):

ag <- aggregate(. ~ ID, DF, function(x) c(mean = mean(x), sd = sd(x)))

Note 1: A commenter pointed out that ag is a data frame for which some columns are matrices. Although initially that may seem strange, in fact it simplifies access. ag has the same number of columns as the input DF. Its first column ag[[1]] is ID and the ith column of the remainder ag[[i+1]] (or equivalanetly ag[-1][[i]]) is the matrix of statistics for the ith input observation column. If one wishes to access the jth statistic of the ith observation it is therefore ag[[i+1]][, j] which can also be written as ag[-1][[i]][, j] .

On the other hand, suppose there are k statistic columns for each observation in the input (where k=2 in the question). Then if we flatten the output then to access the jth statistic of the ith observation column we must use the more complex ag[[k*(i-1)+j+1]] or equivalently ag[-1][[k*(i-1)+j]] .

For example, compare the simplicity of the first expression vs. the second:

ag[-1][[2]]
##        mean      sd
## [1,] 36.333 10.2144
## [2,] 32.250  4.1932
## [3,] 43.500  4.9497

ag_flat <- do.call("data.frame", ag) # flatten
ag_flat[-1][, 2 * (2-1) + 1:2]
##   Obs_2.mean Obs_2.sd
## 1     36.333  10.2144
## 2     32.250   4.1932
## 3     43.500   4.9497

Note 2: The input in reproducible form is:

Lines <- "ID  Obs_1   Obs_2   Obs_3
1   43      48      37
1   27      29      22
1   36      32      40
2   33      38      36
2   29      32      27
2   32      31      35
2   25      28      24
3   45      47      42
3   38      40      36"
DF <- read.table(text = Lines, header = TRUE)

Examples related to r

How to get AIC from Conway–Maxwell-Poisson regression via COM-poisson package in R? R : how to simply repeat a command? session not created: This version of ChromeDriver only supports Chrome version 74 error with ChromeDriver Chrome using Selenium How to show code but hide output in RMarkdown? remove kernel on jupyter notebook Function to calculate R2 (R-squared) in R Center Plot title in ggplot2 R ggplot2: stat_count() must not be used with a y aesthetic error in Bar graph R multiple conditions in if statement What does "The following object is masked from 'package:xxx'" mean?

Examples related to aggregate

Pandas group-by and sum SELECT list is not in GROUP BY clause and contains nonaggregated column Aggregate multiple columns at once Pandas sum by groupby, but exclude certain columns Extract the maximum value within each group in a dataframe How to group dataframe rows into list in pandas groupby Mean per group in a data.frame Summarizing multiple columns with dplyr? data.frame Group By column Compute mean and standard deviation by group for multiple variables in a data.frame

Examples related to reshape

ValueError: cannot reshape array of size 30470400 into shape (50,1104,104) Python reshape list to ndim array Gather multiple sets of columns What does -1 mean in numpy reshape? Grouped bar plot in ggplot Compute mean and standard deviation by group for multiple variables in a data.frame Reshape an array in NumPy How to reshape data from long to wide format load csv into 2D matrix with numpy for plotting Reshaping data.frame from wide to long format

Examples related to reshape2

Grouped bar plot in ggplot R: "Unary operator error" from multiline ggplot2 command Compute mean and standard deviation by group for multiple variables in a data.frame