I would suggest a different approach, using my own library you could do this in a few lines:
var groupMe = sequence(
groupBy(pluck('color')),
forOwn(function(acc, k, v) {
acc.push({colors: k, users: v});
return acc;
},[])
);
var result = groupMe(collection);
This would a be a bit difficult with lodash or Underscore because the arguments are in the opposite order order, so you'd have to use _.partial
a lot.