I wonder whether there is a shortcut to make a simple list out of list of lists in Python.
I can do that in a for loop, but maybe there is some cool "one-liner"? I tried it with reduce(), but I get an error.
Code
l = [[1, 2, 3], [4, 5, 6], [7], [8, 9]]
reduce(lambda x, y: x.extend(y), l)
Error message
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 1, in <lambda>
AttributeError: 'NoneType' object has no attribute 'extend'
This question is related to
python
list
multidimensional-array
flatten
The answer is
You can use itertools.chain():
import itertools
list2d = [[1,2,3], [4,5,6], [7], [8,9]]
merged = list(itertools.chain(*list2d))
Or you can use itertools.chain.from_iterable() which doesn't require unpacking the list with the * operator:
import itertools
list2d = [[1,2,3], [4,5,6], [7], [8,9]]
merged = list(itertools.chain.from_iterable(list2d))
def flatten(l, a):
for i in l:
if isinstance(i, list):
flatten(i, a)
else:
a.append(i)
return a
print(flatten([[[1, [1,1, [3, [4,5,]]]], 2, 3], [4, 5],6], []))
# [1, 1, 1, 3, 4, 5, 2, 3, 4, 5, 6]
Why do you use extend?
reduce(lambda x, y: x+y, l)
This should work fine.
Fastest solution I have found (for large list anyway):
import numpy as np
#turn list into an array and flatten()
np.array(l).flatten()
Done! You can of course turn it back into a list by executing list(l)
def flatten(alist):
if alist == []:
return []
elif type(alist) is not list:
return [alist]
else:
return flatten(alist[0]) + flatten(alist[1:])
I would like to use the concatenate and ravel of the numpy as below
import numpy as np
l = [[1, 2, 3], [4, 5, 6], [7], [8, 9]]
out = np.concatenate(np.array(l)).ravel()
Don't reinvent the wheel if you use Django:
>>> from django.contrib.admin.utils import flatten
>>> l = [[1,2,3], [4,5], [6]]
>>> flatten(l)
>>> [1, 2, 3, 4, 5, 6]
...Pandas:
>>> from pandas.core.common import flatten
>>> list(flatten(l))
...Itertools:
>>> import itertools
>>> flatten = itertools.chain.from_iterable
>>> list(flatten(l))
...Matplotlib
>>> from matplotlib.cbook import flatten
>>> list(flatten(l))
...Unipath:
>>> from unipath.path import flatten
>>> list(flatten(l))
...Setuptools:
>>> from setuptools.namespaces import flatten
>>> list(flatten(l))
I recently came across a situation where I had a mix of strings and numeric data in sublists such as
test = ['591212948',
['special', 'assoc', 'of', 'Chicago', 'Jon', 'Doe'],
['Jon'],
['Doe'],
['fl'],
92001,
555555555,
'hello',
['hello2', 'a'],
'b',
['hello33', ['z', 'w'], 'b']]
where methods like flat_list = [item for sublist in test for item in sublist] have not worked. So, I came up with the following solution for 1+ level of sublists
def concatList(data):
results = []
for rec in data:
if type(rec) == list:
results += rec
results = concatList(results)
else:
results.append(rec)
return results
And the result
In [38]: concatList(test)
Out[38]:
Out[60]:
['591212948',
'special',
'assoc',
'of',
'Chicago',
'Jon',
'Doe',
'Jon',
'Doe',
'fl',
92001,
555555555,
'hello',
'hello2',
'a',
'b',
'hello33',
'z',
'w',
'b']
I personally find it hard to remember all the modules that needed to be imported. Thus I tend to use a simple method, even though I don't know how its performance is compared to other answers.
If you just want to flatten nested lists, then the following will do the job:
def flatten(lst):
for item in lst:
if isinstance(item, list):
yield from flatten(item)
else:
yield item
# test case:
a =[0, [], "fun", [1, 2, 3], [4, [5], 6], 3, [7], [8, 9]]
list(flatten(a))
# output
# [0, 'fun', 1, 2, 3, 4, 5, 6, 3, 7, 8, 9]
However, if you want to flatten a list of iterables (list and/or tuples), it can also do the job with a slight modification:
from collections.abc import Iterable
def flatten(lst):
for item in lst:
if isinstance(item,Iterable) and not isinstance(item,str):
yield from flatten(item)
else:
yield item
# test case:
a =[0, [], "fun", (1, 2, 3), [4, [5], (6)], 3, [7], [8, 9]]
list(flatten(a))
# output:
# [0, 'fun', 1, 2, 3, 4, 5, 6, 3, 7, 8, 9]
Note: Below applies to Python 3.3+ because it uses yield_from. six is also a third-party package, though it is stable. Alternately, you could use sys.version.
In the case of obj = [[1, 2,], [3, 4], [5, 6]], all of the solutions here are good, including list comprehension and itertools.chain.from_iterable.
However, consider this slightly more complex case:
>>> obj = [[1, 2, 3], [4, 5], 6, 'abc', [7], [8, [9, 10]]]
There are several problems here:
- One element,
6, is just a scalar; it's not iterable, so the above routes will fail here. - One element,
'abc', is technically iterable (allstrs are). However, reading between the lines a bit, you don't want to treat it as such--you want to treat it as a single element. - The final element,
[8, [9, 10]]is itself a nested iterable. Basic list comprehension andchain.from_iterableonly extract "1 level down."
You can remedy this as follows:
>>> from collections import Iterable
>>> from six import string_types
>>> def flatten(obj):
... for i in obj:
... if isinstance(i, Iterable) and not isinstance(i, string_types):
... yield from flatten(i)
... else:
... yield i
>>> list(flatten(obj))
[1, 2, 3, 4, 5, 6, 'abc', 7, 8, 9, 10]
Here, you check that the sub-element (1) is iterable with Iterable, an ABC from itertools, but also want to ensure that (2) the element is not "string-like."
This can be done using toolz.concat or cytoolz.concat (cythonized version, that could be faster in some cases):
from cytoolz import concat
l = [[1, 2, 3], [4, 5, 6], [7], [8, 9]]
list(concat(l)) # or just `concat(l)` if one only wants to iterate over the items
On my computer, in python 3.6, this seems to time almost as fast as [item for sublist in l for item in sublist] (not counting the import time):
In [611]: %timeit L = [item for sublist in l for item in sublist]
695 ns ± 2.75 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
In [612]: %timeit L = [item for sublist in l for item in sublist]
701 ns ± 5.5 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
In [613]: %timeit L = list(concat(l))
719 ns ± 12 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
In [614]: %timeit L = list(concat(l))
719 ns ± 22.9 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
The toolz version is indeed slower:
In [618]: from toolz import concat
In [619]: %timeit L = list(concat(l))
845 ns ± 29 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
In [620]: %timeit L = list(concat(l))
833 ns ± 8.73 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
Throwing my hat in the ring...
B = [ [...], [...], ... ]
A = []
for i in B:
A.extend(i)
A bad feature of Anil's function above is that it requires the user to always manually specify the second argument to be an empty list []. This should instead be a default. Due to the way Python objects work, these should be set inside the function, not in the arguments.
Here's a working function:
def list_flatten(l, a=None):
#check a
if a is None:
#initialize with empty list
a = []
for i in l:
if isinstance(i, list):
list_flatten(i, a)
else:
a.append(i)
return a
Testing:
In [2]: lst = [1, 2, [3], [[4]],[5,[6]]]
In [3]: lst
Out[3]: [1, 2, [3], [[4]], [5, [6]]]
In [11]: list_flatten(lst)
Out[11]: [1, 2, 3, 4, 5, 6]
A simple recursive method using reduce from functools and the add operator on lists:
>>> from functools import reduce
>>> from operator import add
>>> flatten = lambda lst: [lst] if type(lst) is int else reduce(add, [flatten(ele) for ele in lst])
>>> flatten(l)
[1, 2, 3, 4, 5, 6, 7, 8, 9]
The function flatten takes in lst as parameter. It loops all the elements of lst until reaching integers (can also change int to float, str, etc. for other data types), which are added to the return value of the outermost recursion.
Recursion, unlike methods like for loops and monads, is that it is a general solution not limited by the list depth. For example, a list with depth of 5 can be flattened the same way as l:
>>> l2 = [[3, [1, 2], [[[6], 5], 4, 0], 7, [[8]], [9, 10]]]
>>> flatten(l2)
[3, 1, 2, 6, 5, 4, 0, 7, 8, 9, 10]
Although it wasn't asked in the original question, there is also an interest in flattenning several times if there are lists of lists of lists...
Also doing at most depth times of un-nesting. Or total flattening (when depth is None), i.e. converting any nesting depth to one global flat list, same as infinite depth.
And supporting varying depth of list-nesting.
Everything that is achieved in my next recursive flatten(l, depth) function and example below, doesn't depend on any module import and works lazily (emits iterator):
def flatten(l, depth = 1):
done, ndepth = False, None
if depth is not None:
done, ndepth = depth <= 0, depth - 1
if not isinstance(l, list):
l, done = [l], True
return iter(l) if done else (e1 for e0 in l for e1 in flatten(e0, ndepth))
l = [ [ [1, [2], 3], [4, 5] ], [ [6, 7], [8, [9, 10]] ] , ['ab', 'c'], 11, 12 ]
for depth in [0, 1, 2, 3, 4, None]:
print('depth', str(depth).rjust(5), ':', list(flatten(l, depth = depth)))
Outputs:
depth 0 : [[[1, [2], 3], [4, 5]], [[6, 7], [8, [9, 10]]], ['ab', 'c'], 11, 12]
depth 1 : [[1, [2], 3], [4, 5], [6, 7], [8, [9, 10]], 'ab', 'c', 11, 12]
depth 2 : [1, [2], 3, 4, 5, 6, 7, 8, [9, 10], 'ab', 'c', 11, 12]
depth 3 : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 'ab', 'c', 11, 12]
depth 4 : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 'ab', 'c', 11, 12]
depth None : [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 'ab', 'c', 11, 12]
I tested most suggested solutions with perfplot (a pet project of mine, essentially a wrapper around timeit), and found
import functools
import operator
functools.reduce(operator.iconcat, a, [])
to be the fastest solution, both when many small lists and few long lists are concatenated. (operator.iadd is equally fast.)
Code to reproduce the plot:
import functools
import itertools
import numpy
import operator
import perfplot
def forfor(a):
return [item for sublist in a for item in sublist]
def sum_brackets(a):
return sum(a, [])
def functools_reduce(a):
return functools.reduce(operator.concat, a)
def functools_reduce_iconcat(a):
return functools.reduce(operator.iconcat, a, [])
def itertools_chain(a):
return list(itertools.chain.from_iterable(a))
def numpy_flat(a):
return list(numpy.array(a).flat)
def numpy_concatenate(a):
return list(numpy.concatenate(a))
perfplot.show(
setup=lambda n: [list(range(10))] * n,
# setup=lambda n: [list(range(n))] * 10,
kernels=[
forfor,
sum_brackets,
functools_reduce,
functools_reduce_iconcat,
itertools_chain,
numpy_flat,
numpy_concatenate,
],
n_range=[2 ** k for k in range(16)],
xlabel="num lists (of length 10)",
# xlabel="len lists (10 lists total)"
)
Recursive version
x = [1,2,[3,4],[5,[6,[7]]],8,9,[10]]
def flatten_list(k):
result = list()
for i in k:
if isinstance(i,list):
#The isinstance() function checks if the object (first argument) is an
#instance or subclass of classinfo class (second argument)
result.extend(flatten_list(i)) #Recursive call
else:
result.append(i)
return result
flatten_list(x)
#result = [1,2,3,4,5,6,7,8,9,10]
from nltk import flatten
l = [[1, 2, 3], [4, 5, 6], [7], [8, 9]]
flatten(l)
The advantage of this solution over most others here is that if you have a list like:
l = [1, [2, 3], [4, 5, 6], [7], [8, 9]]
while most other solutions throw an error this solution handles them.
The reason your function didn't work is because the extend extends an array in-place and doesn't return it. You can still return x from lambda, using something like this:
reduce(lambda x,y: x.extend(y) or x, l)
Note: extend is more efficient than + on lists.
Following seem simplest to me:
>>> import numpy as np
>>> l = [[1, 2, 3], [4, 5, 6], [7], [8, 9]]
>>> print (np.concatenate(l))
[1 2 3 4 5 6 7 8 9]
There are several answers with the same recursive appending scheme as below, but none makes use of try, which makes the solution more robust and pythonic. Other advantages of this solution are:
- works with any kind of iterable (even future ones)
- works with any combination and deepness of nesting
- no dependencies
~
def flatten(itr):
t = tuple()
for e in itr:
try:
t += flatten(e)
except:
t += (e,)
return t
Another variant with same features:
def flatten(itr):
try:
t = sum((flatten(e) for e in itr), tuple())
except:
t = (itr,)
return t
If you need to return a list (which is slightly slower) use list() instead of tuple() and [ ] instead of ( ,).
Consider installing the more_itertools package.
> pip install more_itertools
It ships with an implementation for flatten (source, from the itertools recipes):
import more_itertools
lst = [[1, 2, 3], [4, 5, 6], [7], [8, 9]]
list(more_itertools.flatten(lst))
# [1, 2, 3, 4, 5, 6, 7, 8, 9]
Note: as mentioned in the docs, flatten requires a list of lists. See below on flattening more irregular inputs.
As of version 2.4, you can flatten more complicated, nested iterables with more_itertools.collapse (source, contributed by abarnet).
lst = [[1, 2, 3], [4, 5, 6], [7], [8, 9]]
list(more_itertools.collapse(lst))
# [1, 2, 3, 4, 5, 6, 7, 8, 9]
lst = [[1, 2, 3], [[4, 5, 6]], [[[7]]], 8, 9] # complex nesting
list(more_itertools.collapse(lst))
# [1, 2, 3, 4, 5, 6, 7, 8, 9]
you can use list extend method, it shows to be the fastest:
flat_list = []
for sublist in l:
flat_list.extend(sublist)
performance:
import functools
import itertools
import numpy
import operator
import perfplot
def functools_reduce_iconcat(a):
return functools.reduce(operator.iconcat, a, [])
def itertools_chain(a):
return list(itertools.chain.from_iterable(a))
def numpy_flat(a):
return list(numpy.array(a).flat)
def extend(a):
n = []
list(map(n.extend, a))
return n
perfplot.show(
setup=lambda n: [list(range(10))] * n,
kernels=[
functools_reduce_iconcat, extend,itertools_chain, numpy_flat
],
n_range=[2**k for k in range(16)],
xlabel='num lists',
)
output:
Another unusual approach that works for hetero- and homogeneous lists of integers:
from typing import List
def flatten(l: list) -> List[int]:
"""Flatten an arbitrary deep nested list of lists of integers.
Examples:
>>> flatten([1, 2, [1, [10]]])
[1, 2, 1, 10]
Args:
l: Union[l, Union[int, List[int]]
Returns:
Flatted list of integer
"""
return [int(i.strip('[ ]')) for i in str(l).split(',')]
from functools import reduce #python 3
>>> l = [[1,2,3],[4,5,6], [7], [8,9]]
>>> reduce(lambda x,y: x+y,l)
[1, 2, 3, 4, 5, 6, 7, 8, 9]
The extend() method in your example modifies x instead of returning a useful value (which reduce() expects).
A faster way to do the reduce version would be
>>> import operator
>>> l = [[1,2,3],[4,5,6], [7], [8,9]]
>>> reduce(operator.concat, l)
[1, 2, 3, 4, 5, 6, 7, 8, 9]
If you are willing to give up a tiny amount of speed for a cleaner look, then you could use numpy.concatenate().tolist() or numpy.concatenate().ravel().tolist():
import numpy
l = [[1, 2, 3], [4, 5, 6], [7], [8, 9]] * 99
%timeit numpy.concatenate(l).ravel().tolist()
1000 loops, best of 3: 313 µs per loop
%timeit numpy.concatenate(l).tolist()
1000 loops, best of 3: 312 µs per loop
%timeit [item for sublist in l for item in sublist]
1000 loops, best of 3: 31.5 µs per loop
You can find out more here in the docs numpy.concatenate and numpy.ravel
If you want to flatten a data-structure where you don't know how deep it's nested you could use iteration_utilities.deepflatten1
>>> from iteration_utilities import deepflatten
>>> l = [[1, 2, 3], [4, 5, 6], [7], [8, 9]]
>>> list(deepflatten(l, depth=1))
[1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> l = [[1, 2, 3], [4, [5, 6]], 7, [8, 9]]
>>> list(deepflatten(l))
[1, 2, 3, 4, 5, 6, 7, 8, 9]
It's a generator so you need to cast the result to a list or explicitly iterate over it.
To flatten only one level and if each of the items is itself iterable you can also use iteration_utilities.flatten which itself is just a thin wrapper around itertools.chain.from_iterable:
>>> from iteration_utilities import flatten
>>> l = [[1, 2, 3], [4, 5, 6], [7], [8, 9]]
>>> list(flatten(l))
[1, 2, 3, 4, 5, 6, 7, 8, 9]
Just to add some timings (based on Nico Schlömer answer that didn't include the function presented in this answer):
It's a log-log plot to accommodate for the huge range of values spanned. For qualitative reasoning: Lower is better.
The results show that if the iterable contains only a few inner iterables then sum will be fastest, however for long iterables only the itertools.chain.from_iterable, iteration_utilities.deepflatten or the nested comprehension have reasonable performance with itertools.chain.from_iterable being the fastest (as already noticed by Nico Schlömer).
from itertools import chain
from functools import reduce
from collections import Iterable # or from collections.abc import Iterable
import operator
from iteration_utilities import deepflatten
def nested_list_comprehension(lsts):
return [item for sublist in lsts for item in sublist]
def itertools_chain_from_iterable(lsts):
return list(chain.from_iterable(lsts))
def pythons_sum(lsts):
return sum(lsts, [])
def reduce_add(lsts):
return reduce(lambda x, y: x + y, lsts)
def pylangs_flatten(lsts):
return list(flatten(lsts))
def flatten(items):
"""Yield items from any nested iterable; see REF."""
for x in items:
if isinstance(x, Iterable) and not isinstance(x, (str, bytes)):
yield from flatten(x)
else:
yield x
def reduce_concat(lsts):
return reduce(operator.concat, lsts)
def iteration_utilities_deepflatten(lsts):
return list(deepflatten(lsts, depth=1))
from simple_benchmark import benchmark
b = benchmark(
[nested_list_comprehension, itertools_chain_from_iterable, pythons_sum, reduce_add,
pylangs_flatten, reduce_concat, iteration_utilities_deepflatten],
arguments={2**i: [[0]*5]*(2**i) for i in range(1, 13)},
argument_name='number of inner lists'
)
b.plot()
1 Disclaimer: I'm the author of that library
You can use numpy :
flat_list = list(np.concatenate(list_of_list))
np.hstack(listoflist).tolist()
A funny solution
import pandas as pd
list(pd.DataFrame({'lists':l})['lists'].explode())
[1, 2, 3, 4, 5, 6, 7, 8, 9]
matplotlib.cbook.flatten() will work for nested lists even if they nest more deeply than the example.
import matplotlib
l = [[1, 2, 3], [4, 5, 6], [7], [8, 9]]
print(list(matplotlib.cbook.flatten(l)))
l2 = [[1, 2, 3], [4, 5, 6], [7], [8, [9, 10, [11, 12, [13]]]]]
print list(matplotlib.cbook.flatten(l2))
Result:
[1, 2, 3, 4, 5, 6, 7, 8, 9]
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]
This is 18x faster than underscore._.flatten:
Average time over 1000 trials of matplotlib.cbook.flatten: 2.55e-05 sec
Average time over 1000 trials of underscore._.flatten: 4.63e-04 sec
(time for underscore._)/(time for matplotlib.cbook) = 18.1233394636
I take my statement back. sum is not the winner. Although it is faster when the list is small. But the performance degrades significantly with larger lists.
>>> timeit.Timer(
'[item for sublist in l for item in sublist]',
'l=[[1, 2, 3], [4, 5, 6, 7, 8], [1, 2, 3, 4, 5, 6, 7]] * 10000'
).timeit(100)
2.0440959930419922
The sum version is still running for more than a minute and it hasn't done processing yet!
For medium lists:
>>> timeit.Timer(
'[item for sublist in l for item in sublist]',
'l=[[1, 2, 3], [4, 5, 6, 7, 8], [1, 2, 3, 4, 5, 6, 7]] * 10'
).timeit()
20.126545906066895
>>> timeit.Timer(
'reduce(lambda x,y: x+y,l)',
'l=[[1, 2, 3], [4, 5, 6, 7, 8], [1, 2, 3, 4, 5, 6, 7]] * 10'
).timeit()
22.242258071899414
>>> timeit.Timer(
'sum(l, [])',
'l=[[1, 2, 3], [4, 5, 6, 7, 8], [1, 2, 3, 4, 5, 6, 7]] * 10'
).timeit()
16.449732065200806
Using small lists and timeit: number=1000000
>>> timeit.Timer(
'[item for sublist in l for item in sublist]',
'l=[[1, 2, 3], [4, 5, 6, 7, 8], [1, 2, 3, 4, 5, 6, 7]]'
).timeit()
2.4598159790039062
>>> timeit.Timer(
'reduce(lambda x,y: x+y,l)',
'l=[[1, 2, 3], [4, 5, 6, 7, 8], [1, 2, 3, 4, 5, 6, 7]]'
).timeit()
1.5289170742034912
>>> timeit.Timer(
'sum(l, [])',
'l=[[1, 2, 3], [4, 5, 6, 7, 8], [1, 2, 3, 4, 5, 6, 7]]'
).timeit()
1.0598428249359131
Here is a function using recursion which will work on any arbitrary nested list.
def flatten(nested_lst):
""" Return a list after transforming the inner lists
so that it's a 1-D list.
>>> flatten([[[],["a"],"a"],[["ab"],[],"abc"]])
['a', 'a', 'ab', 'abc']
"""
if not isinstance(nested_lst, list):
return(nested_lst)
res = []
for l in nested_lst:
if not isinstance(l, list):
res += [l]
else:
res += flatten(l)
return(res)
>>> flatten([[[],["a"],"a"],[["ab"],[],"abc"]])
['a', 'a', 'ab', 'abc']
This is a play on the original poster's code. (He wasn't far off)
f = []
list(map(f.extend, l))
This may not be the most efficient way but I thought to put a one-liner (actually a two-liner). Both versions will work on arbitrary hierarchy nested lists, and exploits language features (Python3.5) and recursion.
def make_list_flat (l):
flist = []
flist.extend ([l]) if (type (l) is not list) else [flist.extend (make_list_flat (e)) for e in l]
return flist
a = [[1, 2], [[[[3, 4, 5], 6]]], 7, [8, [9, [10, 11], 12, [13, 14, [15, [[16, 17], 18]]]]]]
flist = make_list_flat(a)
print (flist)
The output is
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18]
This works in a depth first manner. The recursion goes down until it finds a non-list element, then extends the local variable flist and then rolls back it to the parent. Whenever flist is returned, it is extended to the parent's flist in the list comprehension. Therefore, at the root, a flat list is returned.
The above one creates several local lists and returns them which are used to extend the parent's list. I think the way around for this may be creating a gloabl flist, like below.
a = [[1, 2], [[[[3, 4, 5], 6]]], 7, [8, [9, [10, 11], 12, [13, 14, [15, [[16, 17], 18]]]]]]
flist = []
def make_list_flat (l):
flist.extend ([l]) if (type (l) is not list) else [make_list_flat (e) for e in l]
make_list_flat(a)
print (flist)
The output is again
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18]
Although I am not sure at this time about the efficiency.
There seems to be a confusion with operator.add! When you add two lists together, the correct term for that is concat, not add. operator.concat is what you need to use.
If you're thinking functional, it is as easy as this::
>>> from functools import reduce
>>> list2d = ((1, 2, 3), (4, 5, 6), (7,), (8, 9))
>>> reduce(operator.concat, list2d)
(1, 2, 3, 4, 5, 6, 7, 8, 9)
You see reduce respects the sequence type, so when you supply a tuple, you get back a tuple. Let's try with a list::
>>> list2d = [[1, 2, 3],[4, 5, 6], [7], [8, 9]]
>>> reduce(operator.concat, list2d)
[1, 2, 3, 4, 5, 6, 7, 8, 9]
Aha, you get back a list.
How about performance::
>>> list2d = [[1, 2, 3],[4, 5, 6], [7], [8, 9]]
>>> %timeit list(itertools.chain.from_iterable(list2d))
1000000 loops, best of 3: 1.36 µs per loop
from_iterable is pretty fast! But it's no comparison to reduce with concat.
>>> list2d = ((1, 2, 3),(4, 5, 6), (7,), (8, 9))
>>> %timeit reduce(operator.concat, list2d)
1000000 loops, best of 3: 492 ns per loop
Note from the author: This is inefficient. But fun, because monoids are awesome. It's not appropriate for production Python code.
>>> sum(l, [])
[1, 2, 3, 4, 5, 6, 7, 8, 9]
This just sums the elements of iterable passed in the first argument, treating second argument as the initial value of the sum (if not given, 0 is used instead and this case will give you an error).
Because you are summing nested lists, you actually get [1,3]+[2,4] as a result of sum([[1,3],[2,4]],[]), which is equal to [1,3,2,4].
Note that only works on lists of lists. For lists of lists of lists, you'll need another solution.
Simple code for underscore.py package fan
from underscore import _
_.flatten([[1, 2, 3], [4, 5, 6], [7], [8, 9]])
# [1, 2, 3, 4, 5, 6, 7, 8, 9]
It solves all flatten problems (none list item or complex nesting)
from underscore import _
# 1 is none list item
# [2, [3]] is complex nesting
_.flatten([1, [2, [3]], [4, 5, 6], [7], [8, 9]])
# [1, 2, 3, 4, 5, 6, 7, 8, 9]
You can install underscore.py with pip
pip install underscore.py
This works with abitrarily nested lists. It can be easily extended to work with other kinds of iterables.
def flatten(seq):
"""list -> list
return a flattend list from an abitrarily nested list
"""
if not seq:
return []
if not isinstance(seq[0], list):
return [seq[0]] + flatten(seq[1:])
return flatten(seq[0]) + flatten(seq[1:])
Sample run
>>> flatten([1, [2, 3], [[[4, 5, 6], 7], [[8]]], 9])
[1, 2, 3, 4, 5, 6, 7, 8, 9]
You can avoid recursive calls to the stack using an actual stack data structure pretty simply.
alist = [1,[1,2],[1,2,[4,5,6],3, "33"]]
newlist = []
while len(alist) > 0 :
templist = alist.pop()
if type(templist) == type(list()) :
while len(templist) > 0 :
temp = templist.pop()
if type(temp) == type(list()) :
for x in temp :
templist.append(x)
else :
newlist.append(temp)
else :
newlist.append(templist)
print(list(reversed(newlist)))
flat_list = []
for i in list_of_list:
flat_list+=i
This Code also works fine as it just extend the list all the way. Although it is much similar but only have one for loop. So It have less complexity than adding 2 for loops.
One posibility is to treat the array as a string:
elements = [[180.0, 1, 2, 3], [173.8], [164.2], [156.5], [147.2], [138.2]]
list(map(float, str(elements).replace("[", "").replace("]", "").split(",")))
The accepted answer did not work for me when dealing with text-based lists of variable lengths. Here is an alternate approach that did work for me.
l = ['aaa', 'bb', 'cccccc', ['xx', 'yyyyyyy']]
Accepted answer that did not work:
flat_list = [item for sublist in l for item in sublist]
print(flat_list)
['a', 'a', 'a', 'b', 'b', 'c', 'c', 'c', 'c', 'c', 'c', 'xx', 'yyyyyyy']
New proposed solution that did work for me:
flat_list = []
_ = [flat_list.extend(item) if isinstance(item, list) else flat_list.append(item) for item in l if item]
print(flat_list)
['aaa', 'bb', 'cccccc', 'xx', 'yyyyyyy']
One can also use NumPy's flat:
import numpy as np
list(np.array(l).flat)
Edit 11/02/2016: Only works when sublists have identical dimensions.
another fun way to do this:
from functools import reduce
from operator import add
li=[[1,2],[3,4]]
x= reduce(add, li)
Here is a general approach that applies to numbers, strings, nested lists and mixed containers.
Code
#from typing import Iterable
from collections import Iterable # < py38
def flatten(items):
"""Yield items from any nested iterable; see Reference."""
for x in items:
if isinstance(x, Iterable) and not isinstance(x, (str, bytes)):
for sub_x in flatten(x):
yield sub_x
else:
yield x
Notes:
- In Python 3,
yield from flatten(x)can replacefor sub_x in flatten(x): yield sub_x - In Python 3.8, abstract base classes are moved from
collection.abcto thetypingmodule.
Demo
lst = [[1, 2, 3], [4, 5, 6], [7], [8, 9]]
list(flatten(lst)) # nested lists
# [1, 2, 3, 4, 5, 6, 7, 8, 9]
mixed = [[1, [2]], (3, 4, {5, 6}, 7), 8, "9"] # numbers, strs, nested & mixed
list(flatten(mixed))
# [1, 2, 3, 4, 5, 6, 7, 8, '9']
Reference
- This solution is modified from a recipe in Beazley, D. and B. Jones. Recipe 4.14, Python Cookbook 3rd Ed., O'Reilly Media Inc. Sebastopol, CA: 2013.
- Found an earlier SO post, possibly the original demonstration.
Source: Stackoverflow.com