Fastest way to convert an iterator to a list

Question

Having an iterator object  is there something faster  better or more correct than a list comprehension to get a list of the objects returned by the iterator   user list    user for user in user iterator

User · Answer

since python 3 5  you can use   iterable unpacking operator   user list     your iterator    but the pythonic way to do it is   user list    list your iterator

User · Answer

Robino was suggesting to add some tests which make sense  so here is a simple benchmark between 3 possible ways  maybe the most used ones  to convert an iterator to a list   by type constructor  list my iterator   by unpacking    my iterator   using list comprehension   e for e in my iterator  I have been using simple bechmark library from simple benchmark import BenchmarkBuilder from heapq import nsmallest  b   BenchmarkBuilder     b add function   def convert by type constructor size       list iter range size      b add function   def convert by list comprehension size        e for e in iter range size      b add function   def convert by unpacking size         iter range size       b add arguments  Convert an iterator to a list   def argument provider        for exp in range 2  22           size   2  exp         yield size  size  r   b run   r plot     As you can see there is very hard to make a difference between conversion by the constructor and conversion by unpacking  conversion by list comprehension is the    slowest    approach   I have been testing also across different Python versions  3 6  3 7  3 8  3 9  by using the following simple script  import argparse import timeit  parser   argparse ArgumentParser      description  Test convert iterator to list   parser add argument       --size   help  The number of elements from iterator    args   parser parse args    size   int args size  repeat number   10000    do not wait too much if the size is too big if size  gt  10000      repeat number   100   def test convert by type constructor        list iter range size      def test convert by list comprehension         e for e in iter range size      def test convert by unpacking          iter range size      def get avg time in ms func       avg time   timeit timeit func  number repeat number    1000   repeat number     return round avg time  6    funcs    test convert by type constructor           test convert by unpacking  test convert by list comprehension   print  map get avg time in ms  funcs    The script will be executed via a subprocess from a Jupyter Notebook  or a script   the size parameter will be passed through command-line arguments and the script results will be taken from standard output  from subprocess import PIPE  run  import pandas  simple data     constructor        unpacking        comprehension                size        python version         size test   100  1000  10 000  100 000  1 000 000 for version in   3 6    3 7    3 8    3 9        print  test for python   version      for size in size test          command    f python version     perf test convert iterator py   f --size  size            result   run command  stdout PIPE  stderr PIPE  universal newlines True          constructor  unpacking   comprehension   result stdout split                    simple data  constructor   append float constructor           simple data  unpacking   append float unpacking           simple data  comprehension   append float comprehension           simple data  python version   append version          simple data  size   append size   df    pandas DataFrame simple data  df    You can get my full notebook from here  In most of the cases  in my tests  unpacking shows to be faster  but the difference is so small that the results may change from a run to the other  Again  the comprehension approach is the slowest  in fact  the other 2 methods are up to   60  faster

User · Answer

list your iterator

[python] Fastest way to convert an iterator to a list

Examples related to python

Examples related to iterator

Examples related to list-comprehension