This morning I received an email from the Microsoft said that, a meaningful company, Microsoft Open Technologies Inc., had just release their first pre-production, the Redis on Windows.
Redis
If you had been working in the Linux world or interested in NoSQL field you should have probably known or heard about the Redis. Redis is an open source, advanced key-value store. Someone categorized Redis to the distributed cache since it’s an in-memory key-value store. Someone categorized it to the NoSQL, since it provides the feature saving your data into disk. And someone categorized it to the distributed queue, since it supports storing your data into hash and list type and provides the enqueue, dequeue and pub/sub functionalities. You see, Redis is a very powerful tool that can be used widely in building especially a large scaling system, as a distributed cache, NoSQL and message queue.
The original Redis is written in ANSI C, which sponsored by VMWare, works in most POSIX systems like Linux, *BSD, OS X without external dependencies. Linux and OSX are the two operating systems where Redis is developed and more tested, and it’s recommended to use Linux for deploying. As you can see there’s no Windows in the list. Microsoft had submitted a patch into Redis repository to make it available on Windows, but unfortunately Redis doesn’t have any plan to merge it into the main branch. This means all Windows version of Redis are all unofficial, includes this one.
Different from the basic in-memory key-value store, such as the Memcached, the Redis not only provides storing objects by keys, it also can store your data in vary types such as strings, hashes, lists, sets and sorted sets. Redis also have the atomic operations on these types, like appending to a string; incrementing the value in a hash; pushing to a list; computing set intersection, union and difference; or getting the member with highest ranking in a sorted set.
In order to achieve its outstanding performance, Redis works with an in-memory data store. But depending on the use cases, we can persist it either by dumping the dataset to disk every once in a while, or by appending each command to a log. So it looks like a NoSQL database as well. Other features include a simple check-and-set mechanism, pub/sub and configuration settings to make Redis behave like a cache.
Download, Build and Install Redis on Windows
The Redis on Windows now is available on the GitHub of the Microsoft Open Technologies Inc. site. You can download the full source here, which including the Redis Service, Redis Client, Redis Benchmark, etc..
In Windows the simplest way to build the Redis is to use the Visual Studio. If you have installed the C++ component then you can open its solution file under the “msvs” folder. Then just build it.
Navigate to the “msvs\debug” folder there will be a lot of messing files appeared. I don’t have any experience on C and C++ but I can find something useful. The most imortant two files are redis-server.exe and redis-cli.exe.
| redis-server.exe | The execution file that start a Redis server process on your machine. |
| redis-cli.exe | The command line client tool that can operate against a Redis server. |
The simplest way to start a Redis server is just to open a command windows and go to this folder, execute the redis-server.exe then you can see the Redis is now running and on the screen it will show the status every 5 seconds by default.
Very easy, right? If you want to check whether the Redis is working or not, just execute the redis-cli.exe in another command window, and input the Redis command “ping”. If the server you are connecting is fine it will response a “PONG” back to you.
If you had been using the Linux version of Redis before you will find that the execution files, output log and the commands are exactly same. This means you can use any tools and clients you are familiar with which running on the original Redis to this new windows version. For example, let put a key-value pair into the Redis store through the client.
I added two key-value pairs into my Redis by using the SET command, and then searched all keys in the server. It gave me the result on the client screen. And then I got one of them by specifying the key. This is a very common and simple usage. You can find the full command list on this page.
Redis and C#
The Redis on Windows product just ship the windows version of Redis server, without the connection client for the .NET language. But this is not a problem. Since as I mentioned below, the Redis on Windows follows all behaviors and interfaces of the original one. This means we can use any connection client that works with the original Redis to this Windows version.
There are many clients available on this page, includes C, C++, Java, PHP, Ruby, Python, etc.. And of course, the C#. I was using the ServiceStack.Redis as the connection client in one of my previous project. It worked well with my Redis running on Ubuntu, so it should be able to work on the Redis on Windows version as well.
You can download the ServiceStack.Redis here, build it can add the reference of ServiceStack.Interfaces and ServiceStack.Redis into the C# project.
When using the ServiceStack.Redis we need to instant its RedisClient class by specifying the Redis server name (or IP) and port. Then your can execute the Redis commands from vary methods in the RedisClient class. Let’s firstly have a try on how to use the list feature of Redis, which it works like a queue.
First of all, I will invoke the FlushAll method to clear all keys in my Redis. This will remove all items that stored in the server.
1: using (var redis = new RedisClient("127.0.0.1"))
2: {
3: redis.FlushAll();
4: }
Then I will start a thread to let user input something and append it into one of the list (queue) of my Redis. The name of this list would be “default”.
1: var senderThread = new Thread(() =>
2: {
3: using (var redis = new RedisClient("127.0.0.1"))
4: {
5: Console.WriteLine("Input message: ");
6: var message = Console.ReadLine();
7: while (!string.IsNullOrWhiteSpace(message))
8: {
9: redis.EnqueueItemOnList("default", message);
10:
11: Console.WriteLine("Input message: ");
12: message = Console.ReadLine();
13: }
14: }
15: });
16: senderThread.Start();
Next, in the main thread the application will try to dequeue any items from the list, and print the value in the console.
1: using (var redis = new RedisClient("127.0.0.1"))
2: {
3: var result = string.Empty;
4: while (string.Compare("q", result, false) != 0)
5: {
6: result = redis.BlockingDequeueItemFromList("default", TimeSpan.FromSeconds(5));
7: Console.WriteLine("DEQUEUE RESULT = [{0}]", result);
8: }
9: }
The full code would be like this.
1: static void Main(string[] args)
2: {
3: // clean up
4: using (var redis = new RedisClient("127.0.0.1"))
5: {
6: redis.FlushAll();
7: }
8:
9: // enqueue thread definition
10: var senderThread = new Thread(() =>
11: {
12: using (var redis = new RedisClient("127.0.0.1"))
13: {
14: Console.WriteLine("Input message: ");
15: var message = Console.ReadLine();
16: while (!string.IsNullOrWhiteSpace(message))
17: {
18: redis.EnqueueItemOnList("default", message);
19:
20: Console.WriteLine("Input message: ");
21: message = Console.ReadLine();
22: }
23: }
24: });
25: senderThread.Start();
26:
27: // dequeue operation
28: using (var redis = new RedisClient("127.0.0.1"))
29: {
30: var result = string.Empty;
31: while (string.Compare("q", result, false) != 0)
32: {
33: result = redis.BlockingDequeueItemFromList("default", TimeSpan.FromSeconds(5));
34: Console.WriteLine("DEQUEUE RESULT = [{0}]", result);
35: }
36: }
37:
38: Console.WriteLine("Done!");
39: Console.ReadKey();
40: }
As you can see after I input some strings it will be appended into the list, and then dequeued from the list at once.
In this example we only have one thread to dequeue the items. If there are more than one client which is dequeuing the same list, only one of them can retrieve the item. In message bus glossology this is named “queue”, which means only one consumer can retrieve an item. And if there’s no consumer available the items in the queue should be stay there until at least a consumer connected and dequeued.
If it’s a reliable queue, the items in the queue will be remained even though the message bus was terminated or the machine is crashed.
Next, let’s have a try on the pub/sub mode. Pub/Sub mode, also known as the “topic” mode in message bus field. Different from the “queue” mode I mentioned below, in “topic” mode a message will be received by all consumers that is subscribing on the topic. (In Redis it’s said “channel” rather than “topic”.) But if some consumers were not available at that moment the message will be lost and will never be delivered to them even though they got back later.
To use the Pub/Sub in Redis from C# as a consumer we also need to create the RedisClient instance, and use its CreateSubscrption method to bind it to a channel. And when a message comes it will raise the OnMessage event and we can handle it to do our own business logic.
1: static void ConsumerAction(object name)
2: {
3: using (var consumer = new RedisClient("127.0.0.1"))
4: {
5: using (var subscription = consumer.CreateSubscription())
6: {
7: subscription.OnSubscribe = (channel) =>
8: {
9: Console.WriteLine("[{0}] Subscribe to channel '{1}'.", name, channel);
10: };
11: subscription.OnUnSubscribe = (channel) =>
12: {
13: Console.WriteLine("[{0}] Unsubscribe to channel '{1}'.", name, channel);
14: };
15: subscription.OnMessage = (channel, message) =>
16: {
17: Console.WriteLine("[{0}] Received message '{1}' from channel '{2}'.", name, message, channel);
18: };
19:
20: subscription.SubscribeToChannels("default");
21: }
22: }
23: }
In order to demonstrate the behavior of Pub/Sub let’s create there threads as three consumers that are subscribing the same channel.
1: var consumerThread1 = new Thread(new ParameterizedThreadStart(ConsumerAction));
2: var consumerThread2 = new Thread(new ParameterizedThreadStart(ConsumerAction));
3: var consumerThread3 = new Thread(new ParameterizedThreadStart(ConsumerAction));
4: consumerThread1.Start("Consumer 1");
5: consumerThread2.Start("Consumer 2");
6: consumerThread3.Start("Consumer 3");
And use the PublishMessage method of the RedisClient to send some messages in the channel in the main thread. The full code would be like this.
1: static void ConsumerAction(object name)
2: {
3: using (var consumer = new RedisClient("127.0.0.1"))
4: {
5: using (var subscription = consumer.CreateSubscription())
6: {
7: subscription.OnSubscribe = (channel) =>
8: {
9: Console.WriteLine("[{0}] Subscribe to channel '{1}'.", name, channel);
10: };
11: subscription.OnUnSubscribe = (channel) =>
12: {
13: Console.WriteLine("[{0}] Unsubscribe to channel '{1}'.", name, channel);
14: };
15: subscription.OnMessage = (channel, message) =>
16: {
17: Console.WriteLine("[{0}] Received message '{1}' from channel '{2}'.", name, message, channel);
18: };
19:
20: subscription.SubscribeToChannels("default");
21: }
22: }
23: }
24:
25: static void Main(string[] args)
26: {
27: using (var redis = new RedisClient("127.0.0.1"))
28: {
29: redis.FlushAll();
30: }
31:
32: var consumerThread1 = new Thread(new ParameterizedThreadStart(ConsumerAction));
33: var consumerThread2 = new Thread(new ParameterizedThreadStart(ConsumerAction));
34: var consumerThread3 = new Thread(new ParameterizedThreadStart(ConsumerAction));
35: consumerThread1.Start("Consumer 1");
36: consumerThread2.Start("Consumer 2");
37: consumerThread3.Start("Consumer 3");
38:
39: using (var publisher = new RedisClient("127.0.0.1"))
40: {
41: Console.WriteLine("Input message: ");
42: var message = Console.ReadLine();
43: while (!string.IsNullOrWhiteSpace(message))
44: {
45: publisher.PublishMessage("default", message);
46: Console.WriteLine("Input message: ");
47: message = Console.ReadLine();
48: }
49: }
50:
51:
52: Console.WriteLine("Done!");
53: Console.ReadKey();
54: }
Then let’s run our application and input some messages. You can see in this sample all of our three consumers received the messages and process their own business logic.
Summary
Distributed cache, distributed data store and distributed message queue are very important components when we build a large system with high scalability and high performance. Redis is one of the most powerful product. It can be used as a cache, a NoSQL database and message queue as well. And the performance of Redis is outstanding. But for the developers who is working on Windows platform it’s a little pity that Redis doesn’t support Windows by default.
In this post I forwarded the announcement that Microsoft Open Technologies had just published their first un-commercial Redis on Windows version. By using it we can deploy the Redis server on Windows, which means no need to learn on how to build and run it on Linux.
I also demonstrated how to use it from C# through the ServiceStack.Redis client library. Since the Windows version of Redis follows all APIs as the original one we can use anything we are familiar with.
Integrated with my previous post about using the WCF transport extension to build a high scalability system on top of the message bus, since the Redis supports the list data type which implements the message bus feature, we can use it as our underlying transportation as well.
Hope this helps,
Shaun
All documents and related graphics, codes are provided "AS IS" without warranty of any kind.
Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.