[firebase] Cloud Firestore collection count

Is it possible to count how many items a collection has using the new Firebase database, Cloud Firestore?

If so, how do I do that?

In 2020 this is still not available in the Firebase SDK however it is available in Firebase Extensions (Beta) however it's pretty complex to setup and use...

A reasonable approach

Helpers... (create/delete seems redundant but is cheaper than onUpdate)

export const onCreateCounter = () => async (
  change,
  context
) => {
  const collectionPath = change.ref.parent.path;
  const statsDoc = db.doc("counters/" + collectionPath);
  const countDoc = {};
  countDoc["count"] = admin.firestore.FieldValue.increment(1);
  await statsDoc.set(countDoc, { merge: true });
};

export const onDeleteCounter = () => async (
  change,
  context
) => {
  const collectionPath = change.ref.parent.path;
  const statsDoc = db.doc("counters/" + collectionPath);
  const countDoc = {};
  countDoc["count"] = admin.firestore.FieldValue.increment(-1);
  await statsDoc.set(countDoc, { merge: true });
};

export interface CounterPath {
  watch: string;
  name: string;
}

Exported Firestore hooks

export const Counters: CounterPath[] = [
  {
    name: "count_buildings",
    watch: "buildings/{id2}"
  },
  {
    name: "count_buildings_subcollections",
    watch: "buildings/{id2}/{id3}/{id4}"
  }
];


Counters.forEach(item => {
  exports[item.name + '_create'] = functions.firestore
    .document(item.watch)
    .onCreate(onCreateCounter());

  exports[item.name + '_delete'] = functions.firestore
    .document(item.watch)
    .onDelete(onDeleteCounter());
});

In action

The building root collection and all sub collections will be tracked.

Here under the /counters/ root path

Now collection counts will update automatically and eventually! If you need a count, just use the collection path and prefix it with counters.

const collectionPath = 'buildings/138faicnjasjoa89/buildingContacts';
const collectionCount = await db
  .doc('counters/' + collectionPath)
  .get()
  .then(snap => snap.get('count'));

Limitations

As this approach uses a single database and document, it is limited to the Firestore constraint of 1 Update per Second for each counter. It will be eventually consistent, but in cases where large amounts of documents are added/removed the counter will lag behind the actual collection count.

Took me a while to get this working based on some of the answers above, so I thought I'd share it for others to use. I hope it's useful.

'use strict';

const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
const db = admin.firestore();

exports.countDocumentsChange = functions.firestore.document('library/{categoryId}/documents/{documentId}').onWrite((change, context) => {

    const categoryId = context.params.categoryId;
    const categoryRef = db.collection('library').doc(categoryId)
    let FieldValue = require('firebase-admin').firestore.FieldValue;

    if (!change.before.exists) {

        // new document created : add one to count
        categoryRef.update({numberOfDocs: FieldValue.increment(1)});
        console.log("%s numberOfDocs incremented by 1", categoryId);

    } else if (change.before.exists && change.after.exists) {

        // updating existing document : Do nothing

    } else if (!change.after.exists) {

        // deleting document : subtract one from count
        categoryRef.update({numberOfDocs: FieldValue.increment(-1)});
        console.log("%s numberOfDocs decremented by 1", categoryId);

    }

    return 0;
});

Be careful counting number of documents for large collections. It is a little bit complex with firestore database if you want to have a precalculated counter for every collection.

Code like this doesn't work in this case:

export const customerCounterListener = 
    functions.firestore.document('customers/{customerId}')
    .onWrite((change, context) => {

    // on create
    if (!change.before.exists && change.after.exists) {
        return firestore
                 .collection('metadatas')
                 .doc('customers')
                 .get()
                 .then(docSnap =>
                     docSnap.ref.set({
                         count: docSnap.data().count + 1
                     }))
    // on delete
    } else if (change.before.exists && !change.after.exists) {
        return firestore
                 .collection('metadatas')
                 .doc('customers')
                 .get()
                 .then(docSnap =>
                     docSnap.ref.set({
                         count: docSnap.data().count - 1
                     }))
    }

    return null;
});

The reason is because every cloud firestore trigger has to be idempotent, as firestore documentation say: https://firebase.google.com/docs/functions/firestore-events#limitations_and_guarantees

Solution

So, in order to prevent multiple executions of your code, you need to manage with events and transactions. This is my particular way to handle large collection counters:

const executeOnce = (change, context, task) => {
    const eventRef = firestore.collection('events').doc(context.eventId);

    return firestore.runTransaction(t =>
        t
         .get(eventRef)
         .then(docSnap => (docSnap.exists ? null : task(t)))
         .then(() => t.set(eventRef, { processed: true }))
    );
};

const documentCounter = collectionName => (change, context) =>
    executeOnce(change, context, t => {
        // on create
        if (!change.before.exists && change.after.exists) {
            return t
                    .get(firestore.collection('metadatas')
                    .doc(collectionName))
                    .then(docSnap =>
                        t.set(docSnap.ref, {
                            count: ((docSnap.data() && docSnap.data().count) || 0) + 1
                        }));
        // on delete
        } else if (change.before.exists && !change.after.exists) {
            return t
                     .get(firestore.collection('metadatas')
                     .doc(collectionName))
                     .then(docSnap =>
                        t.set(docSnap.ref, {
                            count: docSnap.data().count - 1
                        }));
        }

        return null;
    });

Use cases here:

/**
 * Count documents in articles collection.
 */
exports.articlesCounter = functions.firestore
    .document('articles/{id}')
    .onWrite(documentCounter('articles'));

/**
 * Count documents in customers collection.
 */
exports.customersCounter = functions.firestore
    .document('customers/{id}')
    .onWrite(documentCounter('customers'));

As you can see, the key to prevent multiple execution is the property called eventId in the context object. If the function has been handled many times for the same event, the event id will be the same in all cases. Unfortunately, you must have "events" collection in your database.

Simplest way to do so is to read the size of a "querySnapshot".

db.collection("cities").get().then(function(querySnapshot) {      
    console.log(querySnapshot.size); 
});

You can also read the length of the docs array inside "querySnapshot".

querySnapshot.docs.length;

Or if a "querySnapshot" is empty by reading the empty value, which will return a boolean value.

querySnapshot.empty;

Solution using pagination with offset & limit:

public int collectionCount(String collection) {
        Integer page = 0;
        List<QueryDocumentSnapshot> snaps = new ArrayList<>();
        findDocsByPage(collection, page, snaps);
        return snaps.size();
    }

public void findDocsByPage(String collection, Integer page, 
                           List<QueryDocumentSnapshot> snaps) {
    try {
        Integer limit = 26000;
        FieldPath[] selectedFields = new FieldPath[] { FieldPath.of("id") };
        List<QueryDocumentSnapshot> snapshotPage;
        snapshotPage = fireStore()
                        .collection(collection)
                        .select(selectedFields)
                        .offset(page * limit)
                        .limit(limit)
                        .get().get().getDocuments();    
        if (snapshotPage.size() > 0) {
            snaps.addAll(snapshotPage);
            page++;
            findDocsByPage(collection, page, snaps);
        }
    } catch (InterruptedException | ExecutionException e) {
        e.printStackTrace();
    }
}

• findDocsPage it's a recursive method to find all pages of collection

• selectedFields for otimize query and get only id field instead full body of document

• limit max size of each query page

• page define inicial page for pagination

From the tests I did it worked well for collections with up to approximately 120k records!

I agree with @Matthew, it will cost a lot if you perform such query.

[ADVICE FOR DEVELOPERS BEFORE STARTING THEIR PROJECTS]

Since we have foreseen this situation at the beginning, we can actually make a collection namely counters with a document to store all the counters in a field with type number.

For example:

For each CRUD operation on the collection, update the counter document:

1. When you create a new collection/subcollection: (+1 in the counter) [1 write operation]
2. When you delete a collection/subcollection: (-1 in the counter) [1 write operation]
3. When you update an existing collection/subcollection, do nothing on the counter document: (0)
4. When you read an existing collection/subcollection, do nothing on the counter document: (0)

Next time, when you want to get the number of collection, you just need to query/point to the document field. [1 read operation]

In addition, you can store the collections name in an array, but this will be tricky, the condition of array in firebase is shown as below:

// we send this
['a', 'b', 'c', 'd', 'e']
// Firebase stores this
{0: 'a', 1: 'b', 2: 'c', 3: 'd', 4: 'e'}

// since the keys are numeric and sequential,
// if we query the data, we get this
['a', 'b', 'c', 'd', 'e']

// however, if we then delete a, b, and d,
// they are no longer mostly sequential, so
// we do not get back an array
{2: 'c', 4: 'e'}

So, if you are not going to delete the collection , you can actually use array to store list of collections name instead of querying all the collection every time.

Hope it helps!

As with many questions, the answer is - It depends.

You should be very careful when handling large amounts of data on the front end. On top of making your front end feel sluggish, Firestore also charges you $0.60 per million reads you make.

Small collection (less than 100 documents)

Use with care - Frontend user experience may take a hit

Handling this on the front end should be fine as long as you are not doing too much logic with this returned array.

db.collection('...').get().then(snap => {
   size = snap.size // will return the collection size
});

Medium collection (100 to 1000 documents)

Use with care - Firestore read invocations may cost a lot

Handling this on the front end is not feasible as it has too much potential to slow down the users system. We should handle this logic server side and only return the size.

The drawback to this method is you are still invoking firestore reads (equal to the size of your collection), which in the long run may end up costing you more than expected.

Cloud Function:

...
db.collection('...').get().then(snap => {
    res.status(200).send({length: snap.size});
});

Front End:

yourHttpClient.post(yourCloudFunctionUrl).toPromise().then(snap => {
     size = snap.length // will return the collection size
})

Large collection (1000+ documents)

Most scalable solution

FieldValue.increment()

As of April 2019 Firestore now allows incrementing counters, completely atomically, and without reading the data prior. This ensures we have correct counter values even when updating from multiple sources simultaneously (previously solved using transactions), while also reducing the number of database reads we perform.

By listening to any document deletes or creates we can add to or remove from a count field that is sitting in the database.

See the firestore docs - Distributed Counters Or have a look at Data Aggregation by Jeff Delaney. His guides are truly fantastic for anyone using AngularFire but his lessons should carry over to other frameworks as well.

Cloud Function:

export const documentWriteListener = 
    functions.firestore.document('collection/{documentUid}')
    .onWrite((change, context) => {

    if (!change.before.exists) {
        // New document Created : add one to count

        db.doc(docRef).update({numberOfDocs: FieldValue.increment(1)});
    
    } else if (change.before.exists && change.after.exists) {
        // Updating existing document : Do nothing

    } else if (!change.after.exists) {
        // Deleting document : subtract one from count

        db.doc(docRef).update({numberOfDocs: FieldValue.increment(-1)});

    }

return;
});

Now on the frontend you can just query this numberOfDocs field to get the size of the collection.

var variable=0
variable=variable+querySnapshot.count

then if you are to use it on a String variable then

let stringVariable= String(variable)

UPDATE 11/20

I created an npm package for easy access to a counter function: https://fireblog.io/blog/post/firestore-counters

I created a universal function using all these ideas to handle all counter situations (except queries).

The only exception would be when doing so many writes a second, it slows you down. An example would be likes on a trending post. It is overkill on a blog post, for example, and will cost you more. I suggest creating a separate function in that case using shards: https://firebase.google.com/docs/firestore/solutions/counters

// trigger collections
exports.myFunction = functions.firestore
    .document('{colId}/{docId}')
    .onWrite(async (change: any, context: any) => {
        return runCounter(change, context);
    });

// trigger sub-collections
exports.mySubFunction = functions.firestore
    .document('{colId}/{docId}/{subColId}/{subDocId}')
    .onWrite(async (change: any, context: any) => {
        return runCounter(change, context);
    });

// add change the count
const runCounter = async function (change: any, context: any) {

    const col = context.params.colId;

    const eventsDoc = '_events';
    const countersDoc = '_counters';

    // ignore helper collections
    if (col.startsWith('_')) {
        return null;
    }
    // simplify event types
    const createDoc = change.after.exists && !change.before.exists;
    const updateDoc = change.before.exists && change.after.exists;

    if (updateDoc) {
        return null;
    }
    // check for sub collection
    const isSubCol = context.params.subDocId;

    const parentDoc = `${countersDoc}/${context.params.colId}`;
    const countDoc = isSubCol
        ? `${parentDoc}/${context.params.docId}/${context.params.subColId}`
        : `${parentDoc}`;

    // collection references
    const countRef = db.doc(countDoc);
    const countSnap = await countRef.get();

    // increment size if doc exists
    if (countSnap.exists) {
        // createDoc or deleteDoc
        const n = createDoc ? 1 : -1;
        const i = admin.firestore.FieldValue.increment(n);

        // create event for accurate increment
        const eventRef = db.doc(`${eventsDoc}/${context.eventId}`);

        return db.runTransaction(async (t: any): Promise<any> => {
            const eventSnap = await t.get(eventRef);
            // do nothing if event exists
            if (eventSnap.exists) {
                return null;
            }
            // add event and update size
            await t.update(countRef, { count: i });
            return t.set(eventRef, {
                completed: admin.firestore.FieldValue.serverTimestamp()
            });
        }).catch((e: any) => {
            console.log(e);
        });
        // otherwise count all docs in the collection and add size
    } else {
        const colRef = db.collection(change.after.ref.parent.path);
        return db.runTransaction(async (t: any): Promise<any> => {
            // update size
            const colSnap = await t.get(colRef);
            return t.set(countRef, { count: colSnap.size });
        }).catch((e: any) => {
            console.log(e);
        });;
    }
}

This handles events, increments, and transactions. The beauty in this, is that if you are not sure about the accuracy of a document (probably while still in beta), you can delete the counter to have it automatically add them up on the next trigger. Yes, this costs, so don't delete it otherwise.

Same kind of thing to get the count:

const collectionPath = 'buildings/138faicnjasjoa89/buildingContacts';
const colSnap = await db.doc('_counters/' + collectionPath).get();
const count = colSnap.get('count');

Also, you may want to create a cron job (scheduled function) to remove old events to save money on database storage. You need at least a blaze plan, and there may be some more configuration. You could run it every sunday at 11pm, for example. https://firebase.google.com/docs/functions/schedule-functions

This is untested, but should work with a few tweaks:

exports.scheduledFunctionCrontab = functions.pubsub.schedule('5 11 * * *')
    .timeZone('America/New_York')
    .onRun(async (context) => {

        // get yesterday
        const yesterday = new Date();
        yesterday.setDate(yesterday.getDate() - 1);

        const eventFilter = db.collection('_events').where('completed', '<=', yesterday);
        const eventFilterSnap = await eventFilter.get();
        eventFilterSnap.forEach(async (doc: any) => {
            await doc.ref.delete();
        });
        return null;
    });

And last, don't forget to protect the collections in firestore.rules:

match /_counters/{document} {
  allow read;
  allow write: if false;
}
match /_events/{document} {
  allow read, write: if false;
}

Update: Queries

Adding to my other answer if you want to automate query counts as well, you can use this modified code in your cloud function:

    if (col === 'posts') {

        // counter reference - user doc ref
        const userRef = after ? after.userDoc : before.userDoc;
        // query reference
        const postsQuery = db.collection('posts').where('userDoc', "==", userRef);
        // add the count - postsCount on userDoc
        await addCount(change, context, postsQuery, userRef, 'postsCount');

    }
    return delEvents();

Which will automatically update the postsCount in the userDocument. You could easily add other one to many counts this way. This just gives you ideas of how you can automate things. I also gave you another way to delete the events. You have to read each date to delete it, so it won't really save you to delete them later, just makes the function slower.

/**
 * Adds a counter to a doc
 * @param change - change ref
 * @param context - context ref
 * @param queryRef - the query ref to count
 * @param countRef - the counter document ref
 * @param countName - the name of the counter on the counter document
 */
const addCount = async function (change: any, context: any, 
  queryRef: any, countRef: any, countName: string) {

    // events collection
    const eventsDoc = '_events';

    // simplify event type
    const createDoc = change.after.exists && !change.before.exists;

    // doc references
    const countSnap = await countRef.get();

    // increment size if field exists
    if (countSnap.get(countName)) {
        // createDoc or deleteDoc
        const n = createDoc ? 1 : -1;
        const i = admin.firestore.FieldValue.increment(n);

        // create event for accurate increment
        const eventRef = db.doc(`${eventsDoc}/${context.eventId}`);

        return db.runTransaction(async (t: any): Promise<any> => {
            const eventSnap = await t.get(eventRef);
            // do nothing if event exists
            if (eventSnap.exists) {
                return null;
            }
            // add event and update size
            await t.set(countRef, { [countName]: i }, { merge: true });
            return t.set(eventRef, {
                completed: admin.firestore.FieldValue.serverTimestamp()
            });
        }).catch((e: any) => {
            console.log(e);
        });
        // otherwise count all docs in the collection and add size
    } else {
        return db.runTransaction(async (t: any): Promise<any> => {
            // update size
            const colSnap = await t.get(queryRef);
            return t.set(countRef, { [countName]: colSnap.size }, { merge: true });
        }).catch((e: any) => {
            console.log(e);
        });;
    }
}
/**
 * Deletes events over a day old
 */
const delEvents = async function () {

    // get yesterday
    const yesterday = new Date();
    yesterday.setDate(yesterday.getDate() - 1);

    const eventFilter = db.collection('_events').where('completed', '<=', yesterday);
    const eventFilterSnap = await eventFilter.get();
    eventFilterSnap.forEach(async (doc: any) => {
        await doc.ref.delete();
    });
    return null;
}

I should also warn you that universal functions will run on every onWrite call period. It may be cheaper to only run the function on onCreate and on onDelete instances of your specific collections. Like the noSQL database we are using, repeated code and data can save you money.

This uses counting to create numeric unique ID. In my use, I will not be decrementing ever, even when the document that the ID is needed for is deleted.

Upon a collection creation that needs unique numeric value

1. Designate a collection appData with one document, set with .doc id only
2. Set uniqueNumericIDAmount to 0 in the firebase firestore console
3. Use doc.data().uniqueNumericIDAmount + 1 as the unique numeric id
4. Update appData collection uniqueNumericIDAmount with firebase.firestore.FieldValue.increment(1)

firebase
    .firestore()
    .collection("appData")
    .doc("only")
    .get()
    .then(doc => {
        var foo = doc.data();
        foo.id = doc.id;

        // your collection that needs a unique ID
        firebase
            .firestore()
            .collection("uniqueNumericIDs")
            .doc(user.uid)// user id in my case
            .set({// I use this in login, so this document doesn't
                  // exist yet, otherwise use update instead of set
                phone: this.state.phone,// whatever else you need
                uniqueNumericID: foo.uniqueNumericIDAmount + 1
            })
            .then(() => {

                // upon success of new ID, increment uniqueNumericIDAmount
                firebase
                    .firestore()
                    .collection("appData")
                    .doc("only")
                    .update({
                        uniqueNumericIDAmount: firebase.firestore.FieldValue.increment(
                            1
                        )
                    })
                    .catch(err => {
                        console.log(err);
                    });
            })
            .catch(err => {
                console.log(err);
            });
    });

As far as I know there is no build-in solution for this and it is only possible in the node sdk right now. If you have a

db.collection('someCollection')

you can use

.select([fields])

to define which field you want to select. If you do an empty select() you will just get an array of document references.

example:

db.collection('someCollection').select().get().then( (snapshot) => console.log(snapshot.docs.length) );

This solution is only a optimization for the worst case of downloading all documents and does not scale on large collections!

Also have a look at this:
How to get a count of number of documents in a collection with Cloud Firestore

I have try a lot with different approaches. And finally, I improve one of the methods. First you need to create a separate collection and save there all events. Second you need to create a new lambda to be triggered by time. This lambda will Count events in event collection and clear event documents. Code details in article. https://medium.com/@ihor.malaniuk/how-to-count-documents-in-google-cloud-firestore-b0e65863aeca

There is no direct option available. You cant't do db.collection("CollectionName").count(). Below are the two ways by which you can find the count of number of documents within a collection.

1 :- Get all the documents in the collection and then get it's size.(Not the best Solution)

db.collection("CollectionName").get().subscribe(doc=>{
console.log(doc.size)
})

By using above code your document reads will be equal to the size of documents within a collection and that is the reason why one must avoid using above solution.

2:- Create a separate document with in your collection which will store the count of number of documents in the collection.(Best Solution)

db.collection("CollectionName").doc("counts")get().subscribe(doc=>{
console.log(doc.count)
})

Above we created a document with name counts to store all the count information.You can update the count document in the following way:-

• Create a firestore triggers on the document counts
• Increment the count property of counts document when a new document is created.
• Decrement the count property of counts document when a document is deleted.

w.r.t price (Document Read = 1) and fast data retrieval the above solution is good.

No, there is no built-in support for aggregation queries right now. However there are a few things you could do.

The first is documented here. You can use transactions or cloud functions to maintain aggregate information:

This example shows how to use a function to keep track of the number of ratings in a subcollection, as well as the average rating.

exports.aggregateRatings = firestore
  .document('restaurants/{restId}/ratings/{ratingId}')
  .onWrite(event => {
    // Get value of the newly added rating
    var ratingVal = event.data.get('rating');

    // Get a reference to the restaurant
    var restRef = db.collection('restaurants').document(event.params.restId);

    // Update aggregations in a transaction
    return db.transaction(transaction => {
      return transaction.get(restRef).then(restDoc => {
        // Compute new number of ratings
        var newNumRatings = restDoc.data('numRatings') + 1;

        // Compute new average rating
        var oldRatingTotal = restDoc.data('avgRating') * restDoc.data('numRatings');
        var newAvgRating = (oldRatingTotal + ratingVal) / newNumRatings;

        // Update restaurant info
        return transaction.update(restRef, {
          avgRating: newAvgRating,
          numRatings: newNumRatings
        });
      });
    });
});

The solution that jbb mentioned is also useful if you only want to count documents infrequently. Make sure to use the select() statement to avoid downloading all of each document (that's a lot of bandwidth when you only need a count). select() is only available in the server SDKs for now so that solution won't work in a mobile app.

firebaseFirestore.collection("...").addSnapshotListener(new EventListener<QuerySnapshot>() {
        @Override
        public void onEvent(QuerySnapshot documentSnapshots, FirebaseFirestoreException e) {

            int Counter = documentSnapshots.size();

        }
    });

A workaround is to:

write a counter in a firebase doc, which you increment within a transaction everytime you create a new entry

You store the count in a field of your new entry (i.e: position: 4).

Then you create an index on that field (position DESC).

You can do a skip+limit with a query.Where("position", "<" x).OrderBy("position", DESC)

Hope this helps!

Increment a counter using admin.firestore.FieldValue.increment:

exports.onInstanceCreate = functions.firestore.document('projects/{projectId}/instances/{instanceId}')
  .onCreate((snap, context) =>
    db.collection('projects').doc(context.params.projectId).update({
      instanceCount: admin.firestore.FieldValue.increment(1),
    })
  );

exports.onInstanceDelete = functions.firestore.document('projects/{projectId}/instances/{instanceId}')
  .onDelete((snap, context) =>
    db.collection('projects').doc(context.params.projectId).update({
      instanceCount: admin.firestore.FieldValue.increment(-1),
    })
  );

In this example we increment an instanceCount field in the project each time a document is added to the instances sub collection. If the field doesn't exist yet it will be created and incremented to 1.

The incrementation is transactional internally but you should use a distributed counter if you need to increment more frequently than every 1 second.

It's often preferable to implement onCreate and onDelete rather than onWrite as you will call onWrite for updates which means you are spending more money on unnecessary function invocations (if you update the docs in your collection).