[python] Double Iteration in List Comprehension

In Python you can have multiple iterators in a list comprehension, like

[(x,y) for x in a for y in b]

for some suitable sequences a and b. I'm aware of the nested loop semantics of Python's list comprehensions.

My question is: Can one iterator in the comprehension refer to the other? In other words: Could I have something like this:

[x for x in a for a in b]

where the current value of the outer loop is the iterator of the inner?

As an example, if I have a nested list:

a=[[1,2],[3,4]]

what would the list comprehension expression be to achieve this result:

[1,2,3,4]

?? (Please only list comprehension answers, since this is what I want to find out).

This flatten_nlevel function calls recursively the nested list1 to covert to one level. Try this out

def flatten_nlevel(list1, flat_list):
    for sublist in list1:
        if isinstance(sublist, type(list)):        
            flatten_nlevel(sublist, flat_list)
        else:
            flat_list.append(sublist)

list1 = [1,[1,[2,3,[4,6]],4],5]

items = []
flatten_nlevel(list1,items)
print(items)

output:

[1, 1, 2, 3, 4, 6, 4, 5]

If you want to keep the multi dimensional array, one should nest the array brackets. see example below where one is added to every element.

>>> a = [[1, 2], [3, 4]]

>>> [[col +1 for col in row] for row in a]
[[2, 3], [4, 5]]

>>> [col +1 for row in a for col in row]
[2, 3, 4, 5]

Additionally, you could use just the same variable for the member of the input list which is currently accessed and for the element inside this member. However, this might even make it more (list) incomprehensible.

input = [[1, 2], [3, 4]]
[x for x in input for x in x]

First for x in input is evaluated, leading to one member list of the input, then, Python walks through the second part for x in x during which the x-value is overwritten by the current element it is accessing, then the first x defines what we want to return.

To answer your question with your own suggestion:

>>> [x for b in a for x in b] # Works fine

While you asked for list comprehension answers, let me also point out the excellent itertools.chain():

>>> from itertools import chain
>>> list(chain.from_iterable(a))
>>> list(chain(*a)) # If you're using python < 2.6

Order of iterators may seem counter-intuitive.

Take for example: [str(x) for i in range(3) for x in foo(i)]

Let's decompose it:

def foo(i):
    return i, i + 0.5

[str(x)
    for i in range(3)
        for x in foo(i)
]

# is same as
for i in range(3):
    for x in foo(i):
        yield str(x)

This memory technic helps me a lot:

[ <RETURNED_VALUE> <OUTER_LOOP1> <INNER_LOOP2> <INNER_LOOP3> ... <OPTIONAL_IF> ]

And now you can think about Return + Outer-loop as the only Right Order

Knowing above, the order in list comprehensive even for 3 loops seem easy:

c=[111, 222, 333]
b=[11, 22, 33]
a=[1, 2, 3]

print(
  [
    (i, j, k)                            # <RETURNED_VALUE> 
    for i in a for j in b for k in c     # in order: loop1, loop2, loop3
    if i < 2 and j < 20 and k < 200      # <OPTIONAL_IF>
  ]
)
[(1, 11, 111)]

because the above is just a:

for i in a:                         # outer loop1 GOES SECOND
  for j in b:                       # inner loop2 GOES THIRD
    for k in c:                     # inner loop3 GOES FOURTH
      if i < 2 and j < 20 and k < 200:
        print((i, j, k))            # returned value GOES FIRST

for iterating one nested list/structure, technic is the same: for a from the question:

a = [[1,2],[3,4]]
[i2    for i1 in a      for i2 in i1]
which return [1, 2, 3, 4]

for one another nested level

a = [[[1, 2], [3, 4]], [[5, 6], [7, 8, 9]], [[10]]]
[i3    for i1 in a      for i2 in i1     for i3 in i2]
which return [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

and so on

ThomasH has already added a good answer, but I want to show what happens:

>>> a = [[1, 2], [3, 4]]
>>> [x for x in b for b in a]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'b' is not defined

>>> [x for b in a for x in b]
[1, 2, 3, 4]
>>> [x for x in b for b in a]
[3, 3, 4, 4]

I guess Python parses the list comprehension from left to right. This means, the first for loop that occurs will be executed first.

The second "problem" of this is that b gets "leaked" out of the list comprehension. After the first successful list comprehension b == [3, 4].

I feel this is easier to understand

[row[i] for row in a for i in range(len(a))]

result: [1, 2, 3, 4]

I hope this helps someone else since a,b,x,y don't have much meaning to me! Suppose you have a text full of sentences and you want an array of words.

# Without list comprehension
list_of_words = []
for sentence in text:
    for word in sentence:
       list_of_words.append(word)
return list_of_words

I like to think of list comprehension as stretching code horizontally.

Try breaking it up into:

# List Comprehension 
[word for sentence in text for word in sentence]

Example:

>>> text = (("Hi", "Steve!"), ("What's", "up?"))
>>> [word for sentence in text for word in sentence]
['Hi', 'Steve!', "What's", 'up?']

This also works for generators

>>> text = (("Hi", "Steve!"), ("What's", "up?"))
>>> gen = (word for sentence in text for word in sentence)
>>> for word in gen: print(word)
Hi
Steve!
What's
up?