[python] Iterating on a file doesn't work the second time

I have a problem with iterating on a file. Here's what I type on the interpreter and the result:

>>> f = open('baby1990.html', 'rU')
>>> for line in f.readlines():
...  print(line)
... 
# ... all the lines from the file appear here ...

When I try to iterate on the same open file again I get nothing!

>>> for line in f.readlines():
...  print(line)
... 
>>>

There is no output at all. To solve this I have to close() the file then open it again for reading! Is that normal behavior?

Of course. That is normal and sane behaviour. Instead of closing and re-opening, you could rewind the file.

As the file object reads the file, it uses a pointer to keep track of where it is. If you read part of the file, then go back to it later it will pick up where you left off. If you read the whole file, and go back to the same file object, it will be like reading an empty file because the pointer is at the end of the file and there is nothing left to read. You can use file.tell() to see where in the file the pointer is and file.seek to set the pointer. For example:

>>> file = open('myfile.txt')
>>> file.tell()
0
>>> file.readline()
'one\n'
>>> file.tell()
4L
>>> file.readline()
'2\n'
>>> file.tell()
6L
>>> file.seek(4)
>>> file.readline()
'2\n'

Also, you should know that file.readlines() reads the whole file and stores it as a list. That's useful to know because you can replace:

for line in file.readlines():
    #do stuff
file.seek(0)
for line in file.readlines():
    #do more stuff

with:

lines = file.readlines()
for each_line in lines:
    #do stuff
for each_line in lines:
    #do more stuff

You can also iterate over a file, one line at a time, without holding the whole file in memory (this can be very useful for very large files) by doing:

for line in file:
    #do stuff

The file object is a buffer. When you read from the buffer, that portion that you read is consumed (the read position is shifted forward). When you read through the entire file, the read position is at the end of the file (EOF), so it returns nothing because there is nothing left to read.

If you have to reset the read position on a file object for some reason, you can do:

f.seek(0)