[node.js] What are good message queue options for nodejs?

Looking to use a message queue in a small web app I'm building with node.js. I looked at resque but not sure that's appropriate. The goal is to push notifications to clients based on backend and other client actions with socketio. I could do this with just socketio but I thought maybe a proper message queue would make this cleaner and I wouldn't have to reinvent the wheel.

What are the options out there?

kue is the only message queue you would ever need

Shameless plug: I'm working on Bokeh: a simple, scalable and blazing-fast task queue built on ZeroMQ. It supports pluggable data stores for persisting tasks, currently in-memory, Redis and Riak are supported. Check it out.

You might want to have a look at

Redis Simple Message Queue for Node.js

Which uses Redis and offers most features of Amazons SQS.

How about Azure ServiceBus? It supports nodejs.

Look at node-queue-lib. Perhaps it is enough that you. It support node.js and browsers. Has two delivery strategies: broadcast and round-robin. Only javascript.

Quick example:

var Queue = require('node-queue-lib/queue.core');

var queue = new Queue('Queue name', 'broadcast');

// subscribe on 'Queue name' messages
queue.subscribe(function (err, subscriber) {
    subscriber.on('error', function(err){
        //
    });
    subscriber.on('data', function (data, accept) {
        console.log(data);
        accept(); // accept process message
    });
});

// publish message
queue.publish('test');

You could use the node STOMP client. This would let you integrate with a variety of message queues including:

• ActiveMQ
• RabbitMQ
• HornetQ

I haven't used this library before, so I can't vouch for its quality. But STOMP is a pretty simple protocol so I suspect you can hack it into submission if necessary.

Another option is to use beanstalkd with node. beanstalkd is a very fast "task queue" written in C that is very good if you don't need the feature flexibility of the brokers listed above.

You might also want to check out ewd-qoper8: https://github.com/robtweed/ewd-qoper8

Take a look at node-busmq - it's a production grade, highly available and scalable message bus backed by redis.

I wrote this module for our global cloud and it's currently deployed in our production environment in several datacenters around the world. It supports named queues, peer-to-peer communication, guaranteed delivery and federation.

For more information on why we created this module you can read this blog post: All Aboard The Message Bus

I recommend trying Kestrel, it's fast and simple as Beanstalk but supports fanout queues. Speaks memcached. It's built using Scala and used at Twitter.

I used KUE with socketIO like you described. I stored the socketID with the job and could then retreive it in the Job Complete.. KUE is based on redis and has good examples on github

something like this....

jobs.process('YourQueuedJob',10, function(job, done){
    doTheJob(job, done);
});


function doTheJob(job, done){
    var socket = io.sockets.sockets[job.data.socketId];
    try {
        socket.emit('news', { status : 'completed' , task : job.data.task });
    } catch(err){
        io.sockets.emit('news', { status : 'fail' , task : job.data.task , socketId: job.data.socketId});
    }
    job.complete();
}

Here's a couple of recommendations I can make:

node-amqp: A RabbitMQ client that I have successfully used in combination with Socket.IO to make a real-time multi-player game and chat application amongst other things. Seems reliable enough.

zeromq.node: If you want to go down the non-brokered route this might be worth a look. More work to implement functionality but your more likely to get lower latency and higher throughput.