Python fastest way to create a list of n lists

Question

So I was wondering how to best create a list of blank lists                   Because of how Python works with lists in memory  this doesn t work        n   This does create             but each element is the same list   d        n d 0  append 1     1   1         Something like a list comprehension works   d       for x in xrange 0 n     But this uses the Python VM for looping  Is there any way to use an implied loop  taking advantage of it being written in C    d      map lambda n  d append     xrange 0 10     This is actually slower

User · Answer

The list comprehensions actually are implemented more efficiently than explicit looping (see the dis output for example functions) and the map way has to invoke an ophaque callable object on every iteration, which incurs considerable overhead overhead.

Regardless, [[] for _dummy in xrange(n)] is the right way to do it and none of the tiny (if existent at all) speed differences between various other ways should matter. Unless of course you spend most of your time doing this - but in that case, you should work on your algorithms instead. How often do you create these lists?

User · Answer

The probably only way which is marginally faster than   d       for x in xrange n     is  from itertools import repeat d       for i in repeat None  n     It does not have to create a new int object in every iteration and is about 15   faster on my machine   Edit  Using NumPy  you can avoid the Python loop using  d   numpy empty  n  0   tolist     but this is actually 2 5 times slower than the list comprehension

User · Answer

To create list and list of lists use below syntax       x       for i in range 10     this will create 1-d list and to initialize it put number in   number  and set length of list put length in range length    To create list of lists use below syntax        x      0  for i in range 3   for i in range 10     this will initialize list of lists with 10 3 dimension and with value 0   To access manipulate element        x 1  5  value

User · Answer

So I did some speed comparisons to get the fastest way  List comprehensions are indeed very fast  The only way to get close is to avoid bytecode getting exectuded during construction of the list  My first attempt was the following method  which would appear to be faster in principle   l        for   in range n   l extend map list l      produces a list of length 2  n  of course  This construction is twice as slow as the list comprehension  according to timeit  for both short and long  a million  lists   My second attempt was to use starmap to call the list constructor for me  There is one construction  which appears to run the list constructor at top speed  but still is slower  but only by a tiny amount   from itertools import starmap l   list starmap list       1 lt  lt n      Interesting enough the execution time suggests that it is the final list call that is makes the starmap solution slow  since its execution time is almost exactly equal to the speed of   l   list    for   in range 1 lt  lt n     My third attempt came when I realized that list     also produces a list  so I tried the apperently simple   l   list map list        1 lt  lt n      but this was slower than the starmap call   Conclusion  for the speed maniacs  Do use the list comprehension  Only call functions  if you have to  Use builtins

User · Answer

Here are two methods  one sweet and simple and conceptual   the other more formal and can be extended in a variety of situations  after having read a dataset   Method 1  Conceptual  X2    X1  1 2 3  X2 append X1  X3  4 5 6  X2 append X3  X2 thus has   1 2 3   4 5 6   ie a list of lists     Method 2   Formal and extensible  Another elegant way to store a list as a list of lists of different numbers - which it reads from a file   The file here has the dataset train  Train is a data-set with say 50 rows and 20 columns  ie  Train 0  gives me the 1st row of a csv file  train 1  gives me the 2nd row and so on  I am interested in separating the dataset with 50 rows as one list  except the column 0   which is my explained variable here  so must be removed from the orignal train dataset  and then scaling up list after list- ie a list of a list  Here s the code that does that    Note that I am reading from  1  in the inner loop since I am interested in explanatory variables only  And I re-initialize X1    in the other loop  else the X2 append  0  len train 0  -1    will rewrite X1 over and over again - besides it more memory efficient   X2    for j in range 0 len train        X1        for k in range 1 len train 0             txt2 train j  k          X1 append txt2      X2 append X1 0  len train 0  -1

[python] Python: fastest way to create a list of n lists

Examples related to python