Developing a pubsub application in Redis
Developers use the publish/subscribe (pubsub) messaging model a lot these days to decentralize application and server communications and to reduce single point of failure. With pubsub, one application publishes some data to a digital channel. Applications that are interested in receiving the data can subscribe to the appropriate channel and receive all the data that is published to it. A classic use for this would be a group chat or IRC server, in which users subscribe (or join) channels (or groups) and can chat with other users present in the channels.
Often developers use message queuing applications such as RabbitMQ or ActiveMQ to implement pubsub, but these solutions tend to be resource-intensive. Though Redis supports only a subset of the features of traditional messaging queues, when it comes to pubsub, it works amazingly well and uses few system resources.
Redis is a multipurpose advanced key-value store that provides a high-performance alternative to relational databases. You could also call it a one-of-a-kind data structure server, because it can hold not only plain strings, but also hashes, lists, and other kinds of data as well. For top performance, Redis stores its entire operational data set in memory, but it provides ways for developers to store the data permanently on disk or in a database.
Redis is straightforward to install. Most Linux distributions have precompiled packages available in their software repositories. To install it under CentOS or Red Hat Linux, first add the Extra Packages for Enterprise Linux (EPEL) repository to your system by following the instructions on its FAQ page. Alternatively you can download the EPEL binary that matches the architecture of your machine and the version of your operating system, then install it using the rpm command:
# rpm -ivh epel-release-6-7.noarch.rpm
You can then install Redis with the command
yum install redis.
If the version in the EPEL repository is old, you can install the current Redis version from source:
$ wget http://download.redis.io/releases/redis-2.6.16.tar.gz
$ tar xzf redis-2.6.16.tar.gz
$ cd redis-2.6.16
Start Redis with the command
sudo service redis start if you performed an RPM installation, or
src/redis-server after source installation.
Redis supports many modes of operations, including transaction and pubsub. Developers can use transactional mode when they need atomicity – that is, when they need an entire operation to happen. If one component of an operation fails – updating one of several tables involved, for instance – the entire transaction must be reverted. This is comment with monetary applications, when you debit one account and credit another.
In this article, however, we'll focus on pubsub. Let's build a simple chat application that uses a pubsub architecture. We'll use Python's redis-py library, which you can install with the pip package manager:
$ sudo pip install redis
The application has two components: one publisher and one subscriber. The code for susbcriber.py is simple:
r = redis.Redis("localhost")
for item in ps_obj.listen():
First we import the redis library, which provides pubsub functionality, and Python's system library, which we will use to manage the command-line arguments. We create a Redis object "r" that connects to the Redis server running on localhost. You can use the IP address or domain of any machine running Redis here. Next we create a pubsub object "ps_obj" that subscribes to whatever channel we pass to it as a command-line argument. The object listens to this channel, and we display all the data that we receive from the channel – that is, all the chat messages, creating a group chat effect.
The publisher.py code looks like this:
r = redis.Redis("localhost")
r.publish(sys.argv,"["+sys.argv+"] " + " ".join(sys.argv[3:]))
Here too we import the redis and the system libraries to get the same functionality they provide in the subscriber code. We create a Redis object that connects us to the localhost, and we publish data by using the publish method, which accepts two arguments: the channel name to which data is to be published and the data that is to be published. In the code above, sys.argv represents the channel name and sys.argv and sys.argv constitute the second argument to pass to the publish method.
To publish data to the channel, invoke the code with Python. Let's see how it works by running both scripts in different windows. In the first window, run the subscriber:
$ python subscriber.py test-channel
In the second, run the publisher:
$ python publisher.py test-channel aditya Hi everyone.
$ python publisher.py test-channel aditya This is Aditya.
$ python publisher.py test-channel aditya I like Python and Redis.
The subscriber window will look like this:
[aditya] Hi everyone.
[aditya] This is Aditya.
[aditya] I like Python and Redis.
While a subscriber always needs to be alive in order to receive the message, a publisher can go offline or come online anytime. This is in effect what users do – they come to chat rooms whenever they want and go offline at their convenience.
This is a very simple working example of pubsub using Redis and Python. For production applications, you can use Redis' data persistence features in conjunction with pubsub to ensure minimal or no data loss. Redis also has a cluster feature under development that will allow developers to create a cluster of Redis servers to provide high availability and more scalability.
This work is licensed under a Creative Commons Attribution 3.0 Unported License