[python] IndexError: too many indices for array

I know there is a ton of these threads but all of them are for very simple cases like 3x3 matrices and things of that sort and the solutions do not even begin to apply to my situation. So I'm trying to graph G versus l1 (that's not an eleven, but an L1). The data is in the file that I loaded from an excel file. The excel file is 14x250 so there are 14 arguments, each with 250 data points. I had another user (shout out to Hugh Bothwell!) help me with an error in my code, but now another error has surfaced.

So here is the code in question:

# format for CSV file:
header = ['l1', 'l2', 'l3', 'l4', 'l5', 'EI',
      'S', 'P_right', 'P1_0', 'P3_0',
      'w_left', 'w_right', 'G_left', 'G_right']

def loadfile(filename, skip=None, *args):
    skip = set(skip or [])
    with open(filename, *args) as f:
        cr = csv.reader(f, quoting=csv.QUOTE_NONNUMERIC)
        return np.array(row for i,row in enumerate(cr) if i not in skip)
#plot data
outputs_l1 = [loadfile('C:\\Users\\Chris\\Desktop\\Work\\Python Stuff\\BPCROOM - Shingles analysis\\ERR analysis\\l_1 analysis//BS(1) ERR analysis - l_1 - P_3 = {}.csv'.format(p)) for p in p3_arr]

col = {name:i for i,name in enumerate(header)}

fig = plt.figure()
for data,color in zip(outputs_l1, colors):
    xs  = data[:, col["l1"     ]]
    gl = data[:, col["G_left" ]] * 1000.0    # column 12
    gr = data[:, col["G_right"]] * 1000.0    # column 13
    plt.plot(xs, gl, color + "-", gr, color + "--")
for output, col in zip(outputs_l1, colors):
    plt.plot(output[:,0], output[:,11]*1E3, col+'--')
plt.ticklabel_format(axis='both', style='plain', scilimits=(-1,1))
plt.xlabel('$l1 (m)$')
plt.ylabel('G $(J / m^2) * 10^{-3}$')
plt.xlim(xmin=.2)
plt.ylim(ymax=2, ymin=0)

plt.subplots_adjust(top=0.8, bottom=0.15, right=0.7)

After running the entire program, I recieve the error message:

Traceback (most recent call last):
  File "C:/Users/Chris/Desktop/Work/Python Stuff/New Stuff from Brenday 8 26 2014/CD_ssa_plot(2).py", line 115, in <module>
    xs  = data[:, col["l1"     ]]
IndexError: too many indices for array

and before I ran into that problem, I had another involving the line a few below the one the above error message refers to:

Traceback (most recent call last): File "FILE", line 119, in <module> 
gl = data[:, col["G_left" ]] * 1000.0 # column 12 
IndexError: index 12 is out of bounds for axis 1 with size 12

I understand the first error, but am just having problems fixing it. The second error is confusing for me though. My boss is really breathing down my neck so any help would be GREATLY appreciated!

This question is related to python excel csv error-handling indexing

The answer is


The message that you are getting is not for the default Exception of Python:

For a fresh python list, IndexError is thrown only on index not being in range (even docs say so).

>>> l = []
>>> l[1]
IndexError: list index out of range

If we try passing multiple items to list, or some other value, we get the TypeError:

>>> l[1, 2]
TypeError: list indices must be integers, not tuple

>>> l[float('NaN')]
TypeError: list indices must be integers, not float

However, here, you seem to be using matplotlib that internally uses numpy for handling arrays. On digging deeper through the codebase for numpy, we see:

static NPY_INLINE npy_intp
unpack_tuple(PyTupleObject *index, PyObject **result, npy_intp result_n)
{
    npy_intp n, i;
    n = PyTuple_GET_SIZE(index);
    if (n > result_n) {
        PyErr_SetString(PyExc_IndexError,
                        "too many indices for array");
        return -1;
    }
    for (i = 0; i < n; i++) {
        result[i] = PyTuple_GET_ITEM(index, i);
        Py_INCREF(result[i]);
    }
    return n;
}

where, the unpack method will throw an error if it the size of the index is greater than that of the results.

So, Unlike Python which raises a TypeError on incorrect Indexes, Numpy raises the IndexError because it supports multidimensional arrays.


I think the problem is given in the error message, although it is not very easy to spot:

IndexError: too many indices for array
xs  = data[:, col["l1"     ]]

'Too many indices' means you've given too many index values. You've given 2 values as you're expecting data to be a 2D array. Numpy is complaining because data is not 2D (it's either 1D or None).

This is a bit of a guess - I wonder if one of the filenames you pass to loadfile() points to an empty file, or a badly formatted one? If so, you might get an array returned that is either 1D, or even empty (np.array(None) does not throw an Error, so you would never know...). If you want to guard against this failure, you can insert some error checking into your loadfile function.

I highly recommend in your for loop inserting:

print(data)

This will work in Python 2.x or 3.x and might reveal the source of the issue. You might well find it is only one value of your outputs_l1 list (i.e. one file) that is giving the issue.


Examples related to python

programming a servo thru a barometer Is there a way to view two blocks of code from the same file simultaneously in Sublime Text? python variable NameError Why my regexp for hyphenated words doesn't work? Comparing a variable with a string python not working when redirecting from bash script is it possible to add colors to python output? Get Public URL for File - Google Cloud Storage - App Engine (Python) Real time face detection OpenCV, Python xlrd.biffh.XLRDError: Excel xlsx file; not supported Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation

Examples related to excel

Python: Pandas pd.read_excel giving ImportError: Install xlrd >= 0.9.0 for Excel support Converting unix time into date-time via excel How to increment a letter N times per iteration and store in an array? 'Microsoft.ACE.OLEDB.16.0' provider is not registered on the local machine. (System.Data) How to import an Excel file into SQL Server? Copy filtered data to another sheet using VBA Better way to find last used row Could pandas use column as index? Check if a value is in an array or not with Excel VBA How to sort dates from Oldest to Newest in Excel?

Examples related to csv

Pandas: ValueError: cannot convert float NaN to integer Export result set on Dbeaver to CSV Convert txt to csv python script How to import an Excel file into SQL Server? "CSV file does not exist" for a filename with embedded quotes Save Dataframe to csv directly to s3 Python Data-frame Object has no Attribute (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape How to write to a CSV line by line? How to check encoding of a CSV file

Examples related to error-handling

must declare a named package eclipse because this compilation unit is associated to the named module Error:Failed to open zip file. Gradle's dependency cache may be corrupt What does 'index 0 is out of bounds for axis 0 with size 0' mean? What's the source of Error: getaddrinfo EAI_AGAIN? Error handling with try and catch in Laravel What does "Fatal error: Unexpectedly found nil while unwrapping an Optional value" mean? Raise error in a Bash script Javascript Uncaught TypeError: Cannot read property '0' of undefined Multiple values in single-value context IndexError: too many indices for array

Examples related to indexing

numpy array TypeError: only integer scalar arrays can be converted to a scalar index How to print a specific row of a pandas DataFrame? What does 'index 0 is out of bounds for axis 0 with size 0' mean? How does String.Index work in Swift Pandas KeyError: value not in index Update row values where certain condition is met in pandas Pandas split DataFrame by column value Rebuild all indexes in a Database How are iloc and loc different? pandas loc vs. iloc vs. at vs. iat?