You put the if at the end:
[y for y in a if y not in b]
List comprehensions are written in the same order as their nested full-specified counterparts, essentially the above statement translates to:
outputlist = []
for y in a:
if y not in b:
outputlist.append(y)
Your version tried to do this instead:
outputlist = []
if y not in b:
for y in a:
outputlist.append(y)
but a list comprehension must start with at least one outer loop.