[python] What causes the error "_pickle.UnpicklingError: invalid load key, ' '."?

I'm trying to storage 5000 data elements on an array. This 5000 elements are storage on an existent file (therefore it's not empty).

But I'm getting an error and I don't know what is causing it.

IN:

def array():

    name = 'puntos.df4'

    m = open(name, 'rb')
    v = []*5000

    m.seek(-5000, io.SEEK_END)
    fp = m.tell()
    sz = os.path.getsize(name)

    while fp < sz:
        pt = pickle.load(m)
        v.append(pt)

    m.close()
    return v

OUT:

line 23, in array
pt = pickle.load(m)
_pickle.UnpicklingError: invalid load key, ''.

I had a similar error but with different context when I uploaded a *.p file to Google Drive. I tried to use it later in a Google Colab session, and got this error:

    1 with open("/tmp/train.p", mode='rb') as training_data:
----> 2     train = pickle.load(training_data)
UnpicklingError: invalid load key, '<'.

I solved it by compressing the file, upload it and then unzip on the session. It looks like the pickle file is not saved correctly when you upload/download it so it gets corrupted.

I am not completely sure what you're trying to achieve by seeking to a specific offset and attempting to load individual values manually, the typical usage of the pickle module is:

# save data to a file
with open('myfile.pickle','wb') as fout:
    pickle.dump([1,2,3],fout)

# read data from a file
with open('myfile.pickle') as fin:
    print pickle.load(fin)

# output
>> [1, 2, 3]

If you dumped a list, you'll load a list, there's no need to load each item individually.

you're saying that you got an error before you were seeking to the -5000 offset, maybe the file you're trying to read is corrupted.

If you have access to the original data, I suggest you try saving it to a new file and reading it as in the example.

If you transferred these files through disk or other means, it is likely they were not saved properly.

This may not be relevant to your specific issue, but I had a similar problem when the pickle archive had been created using gzip.

For example if a compressed pickle archive is made like this,

import gzip, pickle
with gzip.open('test.pklz', 'wb') as ofp:
    pickle.dump([1,2,3], ofp)

Trying to open it throws the errors

 with open('test.pklz', 'rb') as ifp:
     print(pickle.load(ifp))
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
_pickle.UnpicklingError: invalid load key, ''.

But, if the pickle file is opened using gzip all is harmonious

with gzip.open('test.pklz', 'rb') as ifp:
    print(pickle.load(ifp))

[1, 2, 3]