[node.js] Populate nested array in mongoose

For someone who has the problem with populate and also wants to do this:

• chat with simple text & quick replies (bubbles)
• 4 database collections for chat: clients, users, rooms, messasges.
• same message DB structure for 3 types of senders: bot, users & clients
• refPath or dynamic reference
• populate with path and model options
• use findOneAndReplace/replaceOne with $exists
• create a new document if the fetched document doesn't exist

CONTEXT

Goal

1. Save a new simple text message to the database & populate it with the user or client data (2 different models).
2. Save a new quickReplies message to the database and populate it with the user or client data.
3. Save each message its sender type: clients, users & bot.
4. Populate only the messages who have the sender clients or users with its Mongoose Models. _sender type client models is clients, for user is users.

Message schema:

const messageSchema = new Schema({
    room: {
        type: Schema.Types.ObjectId,
        ref: 'rooms',
        required: [true, `Room's id`]
    },
    sender: {
         _id: { type: Schema.Types.Mixed },
        type: {
            type: String,
            enum: ['clients', 'users', 'bot'],
            required: [true, 'Only 3 options: clients, users or bot.']
        }
    },
    timetoken: {
        type: String,
        required: [true, 'It has to be a Nanosecond-precision UTC string']
    },
    data: {
        lang: String,
        // Format samples on https://docs.chatfuel.com/api/json-api/json-api
        type: {
            text: String,
            quickReplies: [
                {
                    text: String,
                    // Blocks' ids.
                    goToBlocks: [String]
                }
            ]
        }
    }

mongoose.model('messages', messageSchema);

SOLUTION

My server side API request

My code

Utility function (on chatUtils.js file) to get the type of message that you want to save:

/**
 * We filter what type of message is.
 *
 * @param {Object} message
 * @returns {string} The type of message.
 */
const getMessageType = message => {
    const { type } = message.data;
    const text = 'text',
        quickReplies = 'quickReplies';

    if (type.hasOwnProperty(text)) return text;
    else if (type.hasOwnProperty(quickReplies)) return quickReplies;
};

/**
 * Get the Mongoose's Model of the message's sender. We use
 * the sender type to find the Model.
 *
 * @param {Object} message - The message contains the sender type.
 */
const getSenderModel = message => {
    switch (message.sender.type) {
        case 'clients':
            return 'clients';
        case 'users':
            return 'users';
        default:
            return null;
    }
};

module.exports = {
    getMessageType,
    getSenderModel
};

My server side (using Nodejs) to get the request of saving the message:

app.post('/api/rooms/:roomId/messages/new', async (req, res) => {
        const { roomId } = req.params;
        const { sender, timetoken, data } = req.body;
        const { uuid, state } = sender;
        const { type } = state;
        const { lang } = data;

        // For more info about message structure, look up Message Schema.
        let message = {
            room: new ObjectId(roomId),
            sender: {
                _id: type === 'bot' ? null : new ObjectId(uuid),
                type
            },
            timetoken,
            data: {
                lang,
                type: {}
            }
        };

        // ==========================================
        //          CONVERT THE MESSAGE
        // ==========================================
        // Convert the request to be able to save on the database.
        switch (getMessageType(req.body)) {
            case 'text':
                message.data.type.text = data.type.text;
                break;
            case 'quickReplies':
                // Save every quick reply from quickReplies[].
                message.data.type.quickReplies = _.map(
                    data.type.quickReplies,
                    quickReply => {
                        const { text, goToBlocks } = quickReply;

                        return {
                            text,
                            goToBlocks
                        };
                    }
                );
                break;
            default:
                break;
        }

        // ==========================================
        //           SAVE THE MESSAGE
        // ==========================================
        /**
         * We save the message on 2 ways:
         * - we replace the message type `quickReplies` (if it already exists on database) with the new one.
         * - else, we save the new message.
         */
        try {
            const options = {
                // If the quickRepy message is found, we replace the whole document.
                overwrite: true,
                // If the quickRepy message isn't found, we create it.
                upsert: true,
                // Update validators validate the update operation against the model's schema.
                runValidators: true,
                // Return the document already updated.
                new: true
            };

            Message.findOneAndUpdate(
                { room: roomId, 'data.type.quickReplies': { $exists: true } },
                message,
                options,
                async (err, newMessage) => {
                    if (err) {
                        throw Error(err);
                    }

                    // Populate the new message already saved on the database.
                    Message.populate(
                        newMessage,
                        {
                            path: 'sender._id',
                            model: getSenderModel(newMessage)
                        },
                        (err, populatedMessage) => {
                            if (err) {
                                throw Error(err);
                            }

                            res.send(populatedMessage);
                        }
                    );
                }
            );
        } catch (err) {
            logger.error(
                `#API Error on saving a new message on the database of roomId=${roomId}. ${err}`,
                { message: req.body }
            );

            // Bad Request
            res.status(400).send(false);
        }
    });

TIPs:

For the database:

• Every message is a document itself.
• Instead of using refPath, we use the util getSenderModel that is used on populate(). This is because of the bot. The sender.type can be: users with his database, clients with his database and bot without a database. The refPath needs true Model reference, if not, Mongooose throw an error.
• sender._id can be type ObjectId for users and clients, or null for the bot.

For API request logic:

• We replace the quickReply message (Message DB has to have only one quickReply, but as many simple text messages as you want). We use the findOneAndUpdate instead of replaceOne or findOneAndReplace.
• We execute the query operation (the findOneAndUpdate) and the populate operation with the callback of each one. This is important if you don't know if use async/await, then(), exec() or callback(err, document). For more info look the Populate Doc.
• We replace the quick reply message with the overwrite option and without $set query operator.
• If we don't find the quick reply, we create a new one. You have to tell to Mongoose this with upsert option.
• We populate only one time, for the replaced message or the new saved message.
• We return to callbacks, whatever is the message we've saved with findOneAndUpdate and for the populate().
• In populate, we create a custom dynamic Model reference with the getSenderModel. We can use the Mongoose dynamic reference because the sender.type for bot hasn't any Mongoose Model. We use a Populating Across Database with model and path optins.

I've spend a lot of hours solving little problems here and there and I hope this will help someone!