Convert a list to a data frame

Question

I have a nested list of data  Its length is 132 and each item is a list of length 20  Is there a quick way to convert this structure into a data frame that has 132 rows and 20 columns of data  Here is some sample data to work with  l  lt - replicate    132    as list sample letters  20      simplify   FALSE

User · Answer

This is what finally worked for me   do call  rbind   lapply S1  as data frame

User · Answer

Sometimes your data may be a list of lists of vectors of the same length   lolov   list list c 1 2 3  c 4 5 6    list c 7 8 9  c 10 11 12  c 13 14 15        The inner vectors could also be lists  but I m simplifying to make this easier to read    Then you can make the following modification  Remember that you can unlist one level at a time   lov   unlist lolov  recursive   FALSE    gt  lov   1    1  1 2 3    2    1  4 5 6    3    1  7 8 9    4    1  10 11 12    5    1  13 14 15   Now use your favorite method mentioned in the other answers   library plyr   gt ldply lov    V1 V2 V3 1  1  2  3 2  4  5  6 3  7  8  9 4 10 11 12 5 13 14 15

User · Answer

Fixing the sample data so it matches the original description  each item is a list of length 20  mylistlist  lt - replicate    132    as list sample letters  20      simplify   FALSE    we can convert it to a data frame like this  data frame t sapply mylistlist c     sapply converts it to a matrix  data frame converts the matrix to a data frame  resulting in

User · Answer

For a paralleled  multicore  multisession  etc  solution using purrr family of solutions  use   library  furrr  plan multisession    see below to see which other plan   is the more efficient myTibble  lt - future map dfc l    x    Where l is the list   To benchmark the most efficient plan   you can use   library tictoc  plan sequential    reference time   plan multisession    benchamark plan   goes here  See  plan    tic   myTibble  lt - future map dfc l    x  toc

User · Answer

Reshape2 yields the same output as the plyr example above   library reshape2  l  lt - list a   list var 1   1  var 2   2  var 3   3              b   list var 1   4  var 2   5  var 3   6              c   list var 1   7  var 2   8  var 3   9              d   list var 1   10  var 2   11  var 3   12    l  lt - melt l  dcast l  L1   L2    yields     L1 var 1 var 2 var 3 1  a     1     2     3 2  b     4     5     6 3  c     7     8     9 4  d    10    11    12   If you were almost out of pixels you could do this all in 1 line w  recast

User · Answer

With rbind  do call rbind data frame  your list    Edit  Previous version return data frame of list s instead of vectors  as  IanSudbery pointed out in comments

User · Answer

A short  but perhaps not the fastest  way to do this would be to use base r  since a data frame is just a list of equal length vectors  Thus the conversion between your input list and a 30 x 132 data frame would be   df  lt - data frame l    From there we can transpose it to a 132 x 30 matrix  and convert it back to a dataframe   new df  lt - data frame t df     As a one-liner   new df  lt - data frame t data frame l      The rownames will be pretty annoying to look at  but you could always rename those with  rownames new df   lt - 1 nrow new df

User · Answer

Try collapse  unlist2d  shorthand for  unlist to data frame    l  lt - replicate    132    list sample letters  20      simplify   FALSE    library collapse  head unlist2d l      id 1  id 2 V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13 V14 V15 V16 V17 V18 V19 V20 1     1     1  e  x  b  d  s  p  a  c  k   z   q   m   u   l   h   n   r   t   o   y 2     2     1  r  t  i  k  m  b  h  n  s   e   p   f   o   c   x   l   g   v   a   j 3     3     1  t  r  v  z  a  u  c  o  w   f   m   b   d   g   p   q   y   e   n   k 4     4     1  x  i  e  p  f  d  q  k  h   b   j   s   z   a   t   v   y   l   m   n 5     5     1  d  z  k  y  a  p  b  h  c   v   f   m   u   l   n   q   e   i   w   j 6     6     1  l  f  s  u  o  v  p  z  q   e   r   c   h   n   a   t   m   k   y   x  head unlist2d l  idcols   FALSE     V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13 V14 V15 V16 V17 V18 V19 V20 1  e  x  b  d  s  p  a  c  k   z   q   m   u   l   h   n   r   t   o   y 2  r  t  i  k  m  b  h  n  s   e   p   f   o   c   x   l   g   v   a   j 3  t  r  v  z  a  u  c  o  w   f   m   b   d   g   p   q   y   e   n   k 4  x  i  e  p  f  d  q  k  h   b   j   s   z   a   t   v   y   l   m   n 5  d  z  k  y  a  p  b  h  c   v   f   m   u   l   n   q   e   i   w   j 6  l  f  s  u  o  v  p  z  q   e   r   c   h   n   a   t   m   k   y   x

User · Answer

Extending on  Marek s answer  if you want to avoid strings to be turned into factors and efficiency is not a concern try  do call rbind  lapply your list  data frame  stringsAsFactors FALSE

User · Answer

Every solution I have found seems to only apply when every object in a list has the same length   I needed to convert a list to a data frame when the length of the objects in the list were of unequal length   Below is the base R solution I came up with   It no doubt is very inefficient  but it does seem to work  x1  lt - c 2  13  x2  lt - c 2  4  6  9  11  13  x3  lt - c 1  1  2  3  3  4  5  5  6  7  7  8  9  9  10  11  11  12  13  13  my results  lt - list x1  x2  x3     identify length of each list my lengths  lt - unlist lapply my results  function  x    length unlist x      my lengths   1   2  6 20    create a vector of values in all lists my values  lt - as numeric unlist c do call rbind  lapply my results  as data frame      my values   1   2 13  2  4  6  9 11 13  1  1  2  3  3  4  5  5  6  7  7  8  9  9 10 11 11 12 13 13  my matrix  lt - matrix NA  nrow   max my lengths   ncol   length my lengths    my cumsum  lt - cumsum my lengths   mm  lt - 1  for i in 1 length my lengths           my matrix 1 my lengths i  i   lt - my values mm my cumsum i         mm  lt - my cumsum i  1     my df  lt - as data frame my matrix  my df     V1 V2 V3  1   2  2  1  2  13  4  1  3  NA  6  2  4  NA  9  3  5  NA 11  3  6  NA 13  4  7  NA NA  5  8  NA NA  5  9  NA NA  6  10 NA NA  7  11 NA NA  7  12 NA NA  8  13 NA NA  9  14 NA NA  9  15 NA NA 10  16 NA NA 11  17 NA NA 11  18 NA NA 12  19 NA NA 13  20 NA NA 13

User · Answer

The following simple command worked for me   myDf  lt - as data frame myList    Reference  Quora answer    gt  myList  lt - list a   c 1  2  3   b   c 4  5  6    gt  myList  a  1  1 2 3   b  1  4 5 6   gt  myDf  lt - as data frame myList    a b 1 1 4 2 2 5 3 3 6  gt  class myDf   1   data frame    But this will fail if it   s not obvious how to convert the list to a data frame    gt  myList  lt - list a   c 1  2  3   b   c 4  5  6  7    gt  myDf  lt - as data frame myList  Error in  function       row names   NULL  check rows   FALSE  check names   TRUE        arguments imply differing number of rows  3  4   Note  The answer is toward the title of the question and may skips some details of the question

User · Answer

assume your list is called L   data frame Reduce rbind  L

User · Answer

For the general case of deeply nested lists with 3 or more levels like the ones obtained from a nested JSON      2015        spain     population   43   GNP   9      sweden     population   7   GNP   6     2016        spain     population   45   GNP   10      sweden     population   9   GNP   8       consider the approach of melt   to convert the nested list to a tall format first    myjson  lt - jsonlite fromJSON file  test json    tall  lt - reshape2  melt myjson    c  L1    L2    L3    value        L1     L2         L3 value 1 2015  spain population    43 2 2015  spain        GNP     9 3 2015 sweden population     7 4 2015 sweden        GNP     6 5 2016  spain population    45 6 2016  spain        GNP    10 7 2016 sweden population     9 8 2016 sweden        GNP     8   followed by dcast   then to wide again into a tidy dataset where each variable forms a a column and each observation forms a row    wide  lt - reshape2  dcast tall  L1 L2 L3     left side of the formula defines the rows observations and the    right side defines the variables measurements     L1     L2 GNP population 1 2015  spain   9         43 2 2015 sweden   6          7 3 2016  spain  10         45 4 2016 sweden   8          9

User · Answer

l  lt - replicate 10 list sample letters  20    a  lt -lapply l 1 10  data frame  do call  cbind   a

User · Answer

This method uses a tidyverse package  purrr    The list   x  lt - as list mtcars    Converting it into a data frame  a tibble more specifically    library purrr  map df x    x

User · Answer

Depending on the structure of your lists there are some tidyverse options that work nicely with unequal length lists   l  lt - list a   list var 1   1  var 2   2  var 3   3            b   list var 1   4  var 2   5            c   list var 1   7  var 3   9            d   list var 1   10  var 2   11  var 3   NA    df  lt - dplyr  bind rows l  df  lt - purrr  map df l  dplyr  bind rows  df  lt - purrr  map df l    x     all create the same data frame    A tibble  4 x 3   var 1 var 2 var 3    lt dbl gt   lt dbl gt   lt dbl gt  1     1     2     3 2     4     5    NA 3     7    NA     9 4    10    11    NA   You can also mix vectors and data frames   library dplyr  bind rows    list a   1  b   2     data frame a   3 4  b   5 6     c a   7       A tibble  4 x 2       a     b    lt dbl gt   lt dbl gt  1     1     2 2     3     5 3     4     6 4     7    NA

User · Answer

How about using map  function together with a for loop  Here is my solution  list to df  lt - function list to convert      tmp data frame  lt - data frame     for  i in 1 length list to convert         tmp  lt - map dfr list to convert  i    data frame      tmp data frame  lt - rbind tmp data frame  tmp        return tmp data frame     where map dfr convert each of the list element into a data frame and then rbind union them altogether  In your case  I guess it would be  converted list  lt - list to df l

User · Answer

More answers  along with timings in the answer to this question  What is the most efficient way to cast a list as a data frame   The quickest way  that doesn t produce a dataframe with lists rather than vectors for columns appears to be  from Martin Morgan s answer    l  lt - list list col1  a  col2 1  list col1  b  col2 2   f   function x  function i  unlist lapply x        i   use names FALSE  as data frame Map f l   names l  1

User · Answer

The tibble package has a function enframe   that solves this problem by coercing nested list objects to nested tibble   tidy  data frame  objects  Here s a brief example from R for Data Science   x  lt - list      a   1 5      b   3 4       c   5 6     df  lt - enframe x  df   gt    A tibble  3    2   gt     name     value   gt     lt chr gt      lt list gt    gt     1     a  lt int  5  gt    gt     2     b  lt int  2  gt    gt     3     c  lt int  2  gt    Since you have several nests in your list  l  you can use the unlist recursive   FALSE  to remove unnecessary nesting to get just a single hierarchical list and then pass to enframe    I use tidyr  unnest   to unnest the output into a single level  tidy  data frame  which has your two columns  one for the group name and one for the observations with the groups value   If you want columns that make wide  you can add a column using add column   that just repeats the order of the values 132 times  Then just spread   the values      library tidyverse   l  lt - replicate      132      list sample letters  20        simplify   FALSE    l tib  lt - l   gt        unlist recursive   FALSE    gt        enframe     gt        unnest   l tib   gt    A tibble  2 640 x 2   gt      name value   gt      lt int gt   lt chr gt    gt  1      1     d   gt  2      1     z   gt  3      1     l   gt  4      1     b   gt  5      1     i   gt  6      1     j   gt  7      1     g   gt  8      1     w   gt  9      1     r   gt  10     1     p   gt        with 2 630 more rows  l tib spread  lt - l tib   gt       add column index   rep 1 20  132     gt       spread key   index  value   value  l tib spread   gt    A tibble  132 x 21   gt      name    1     2     3     4     5     6     7     8     9    10    11    gt      lt int gt   lt chr gt   lt chr gt   lt chr gt   lt chr gt   lt chr gt   lt chr gt   lt chr gt   lt chr gt   lt chr gt   lt chr gt   lt chr gt    gt  1      1     d     z     l     b     i     j     g     w     r     p     y   gt  2      2     w     s     h     r     i     k     d     u     a     f     j   gt  3      3     r     v     q     s     m     u     j     p     f     a     i   gt  4      4     o     y     x     n     p     i     f     m     h     l     t   gt  5      5     p     w     v     d     k     a     l     r     j     q     n   gt  6      6     i     k     w     o     c     n     m     b     v     e     q   gt  7      7     c     d     m     i     u     o     e     z     v     g     p   gt  8      8     f     s     e     o     p     n     k     x     c     z     h   gt  9      9     d     g     o     h     x     i     c     y     t     f     j   gt  10    10     y     r     f     k     d     o     b     u     i     x     s   gt        with 122 more rows  and 9 more variables   12   lt chr gt    13   lt chr gt     gt       14   lt chr gt    15   lt chr gt    16   lt chr gt    17   lt chr gt    18   lt chr gt     gt       19   lt chr gt    20   lt chr gt

User · Answer

Update July 2020  The default for the parameter stringsAsFactors is now default stringsAsFactors   which in turn yields FALSE as its default   Assuming your list of lists is called l  df  lt - data frame matrix unlist l   nrow length l   byrow TRUE    The above will convert all character columns to factors  to avoid this you can add a parameter to the data frame   call  df  lt - data frame matrix unlist l   nrow 132  byrow TRUE  stringsAsFactors FALSE

User · Answer

You can use the plyr package  For example a nested list of the form  l  lt - list a   list var 1   1  var 2   2  var 3   3          b   list var 1   4  var 2   5  var 3   6          c   list var 1   7  var 2   8  var 3   9          d   list var 1   10  var 2   11  var 3   12            has now a length of 4 and each list in l contains another list of the length 3  Now you can run    library  plyr    df  lt - ldply  l  data frame    and should get the same result as in the answer  Marek and  nico

User · Answer

The package data table has the function rbindlist which is a superfast implementation of do call rbind  list         It can take a list of  lists  data frames or data tables  as input   library data table  ll  lt - list a   list var 1   1  var 2   2  var 3   3      b   list var 1   4  var 2   5  var 3   6      c   list var 1   7  var 2   8  var 3   9      d   list var 1   10  var 2   11  var 3   12       DT  lt - rbindlist ll    This returns a data table inherits from data frame   If you really want to convert back to a data frame use as data frame DT

[r] Convert a list to a data frame

Examples related to r

Examples related to list

Examples related to dataframe