[python] Get class labels from Keras functional model

I have a functional model in Keras (Resnet50 from repo examples). I trained it with ImageDataGenerator and flow_from_directory data and saved model to .h5 file. When I call model.predict I get an array of class probabilities. But I want to associate them with class labels (in my case - folder names). How can I get them? I found that I could use model.predict_classes and model.predict_proba, but I don't have these functions in Functional model, only in Sequential.

It is possible to save a "list" of labels in keras model directly. This way the user who uses the model for predictions and does not have any other sources of information can perform the lookup himself. Here is a dummy example of how one can perform an "injection" of labels

# assume we get labels as list
labels = ["cat","dog","horse","tomato"]
# here we start building our model with input image 299x299 and one output layer
xx = Input(shape=(299,299,3))
flat = Flatten()(xx)
output = Dense(shape=(4))(flat)
# here we perform injection of labels
tf_labels = tf.constant([labels],dtype="string")
tf_labels = tf.tile(labels,[tf.shape(xx)[0],1])
output_labels = Lambda(lambda x: tf_labels,name="label_injection")(xx)
#and finaly creating a model
model=tf.keras.Model(xx,[output,output_labels])

When used for prediction, this model returns tensor of scores and tensot of string labels. Model like this can be saved to h5. In this case the file contains the labels. This model can also be exported to saved_model and used for serving in the cloud.

You must use the labels index you have, here what I do for text classification:

# data labels = [1, 2, 1...]
labels_index = { "website" : 0, "money" : 1 ....} 
# to feed model
label_categories = to_categorical(np.asarray(labels)) 

Then, for predictions:

texts = ["hello, rejoins moi sur skype", "bonjour comment ça va ?", "tu me donnes de l'argent"]

sequences = tokenizer.texts_to_sequences(texts)

data = pad_sequences(sequences, maxlen=MAX_SEQUENCE_LENGTH)

predictions = model.predict(data)

t = 0

for text in texts:
    i = 0
    print("Prediction for \"%s\": " % (text))
    for label in labels_index:
        print("\t%s ==> %f" % (label, predictions[t][i]))
        i = i + 1
    t = t + 1

This gives:

Prediction for "hello, rejoins moi sur skype": 
    website ==> 0.759483
    money ==> 0.037091
    under ==> 0.010587
    camsite ==> 0.114436
    email ==> 0.075975
    abuse ==> 0.002428
Prediction for "bonjour comment ça va ?": 
    website ==> 0.433079
    money ==> 0.084878
    under ==> 0.048375
    camsite ==> 0.036674
    email ==> 0.369197
    abuse ==> 0.027798
Prediction for "tu me donnes de l'argent": 
    website ==> 0.006223
    money ==> 0.095308
    under ==> 0.003586
    camsite ==> 0.003115
    email ==> 0.884112
    abuse ==> 0.007655

In addition to @Emilia Apostolova answer to get the ground truth labels, from

generator = train_datagen.flow_from_directory("train", batch_size=batch_size)

just call

y_true_labels = generator.classes

y_prob = model.predict(x) 
y_classes = y_prob.argmax(axis=-1)

As suggested here.

To map predicted classes and filenames using ImageDataGenerator, I use:

# Data generator and prediction
test_datagen = ImageDataGenerator(rescale=1./255)
test_generator = test_datagen.flow_from_directory(
        inputpath,
        target_size=(150, 150),
        batch_size=20,
        class_mode='categorical',
        shuffle=False)
pred = model.predict_generator(test_generator, steps=len(test_generator), verbose=0)
# Get classes by max element in np (as a list)
classes = list(np.argmax(pred, axis=1))
# Get filenames (set shuffle=false in generator is important)
filenames = test_generator.filenames

I can loop over predicted classes and the associated filename using:

for f in zip(classes, filenames):
    ...

UPDATE: This is no longer valid for newer Keras versions. Please use argmax() as in the answer from Emilia Apostolova.

The functional API models have just the predict() function which for classification would return the class probabilities. You can then select the most probable classes using the probas_to_classes() utility function. Example:

y_proba = model.predict(x)
y_classes = keras.np_utils.probas_to_classes(y_proba)

This is equivalent to model.predict_classes(x) on the Sequential model.

The reason for this is that the functional API support more general class of tasks where predict_classes() would not make sense.

More info: https://github.com/fchollet/keras/issues/2524

When one uses flow_from_directory the problem is how to interpret the probability outputs. As in, how to map the probability outputs and the class labels as how flow_from_directory creates one-hot vectors is not known in prior.

We can get a dictionary that maps the class labels to the index of the prediction vector that we get as the output when we use

generator= train_datagen.flow_from_directory("train", batch_size=batch_size)
label_map = (generator.class_indices)

The label_map variable is a dictionary like this

{'class_14': 5, 'class_10': 1, 'class_11': 2, 'class_12': 3, 'class_13': 4, 'class_2': 6, 'class_3': 7, 'class_1': 0, 'class_6': 10, 'class_7': 11, 'class_4': 8, 'class_5': 9, 'class_8': 12, 'class_9': 13}

Then from this the relation can be derived between the probability scores and class names.

Basically, you can create this dictionary by this code.

from glob import glob
class_names = glob("*") # Reads all the folders in which images are present
class_names = sorted(class_names) # Sorting them
name_id_map = dict(zip(class_names, range(len(class_names))))

The variable name_id_map in the above code also contains the same dictionary as the one obtained from class_indices function of flow_from_directory.

Hope this helps!