If you just want to iterate over the tokens, this is pretty neat:
line = "one, two and 3!"
for token in string.gmatch(line, "[^%s]+") do
print(token)
end
Output:
one,
two
and
3!
Short explanation: the "[^%s]+" pattern matches to every non-empty string in between space characters.