March 30, 2025

Setup a Redis Cluster using Redis Stack

Bill018 mins
Setup a Redis Cluster using Redis Stack


Redis® Cluster is a fully distributed implementation with automated sharding capabilities (horizontal scaling capabilities), designed for high performance and linear scaling up to 1000 nodes. .

This article shows how to set up a Redis Cluster using Redis Stack.

Image description



Prerequisites

Linux preinstalled servers, e.g.:

Host name IP
redis-1 10.0.0.124
redis-2 10.0.0.125
redis-3 10.0.0.126

Cluster topology:

redis-1 redis-2 redis-3
Primary M1 (port 7000) Primary M2 (port 7000) Primary M3 (port 7000)
Replica S3 (port 7001) Replica S1 (port 7001) Replica S2 (port 7001)



Setup Redis Cluster



Install and configure three Redis nodes of the cluster

  1. Connect to all hosts.

  2. Create the file /etc/yum.repos.d/redis.repo with the following contents:

    [Redis]
name=Redis
baseurl=http://packages.redis.io/rpm/rhel9 # replace rhel7 with the appropriate value for your platform
enabled=1
gpgcheck=1

  3. Run the following commands to install Redis Stack:

    curl -fsSL https://packages.redis.io/gpg > /tmp/redis.key
sudo rpm --import /tmp/redis.key
sudo yum install epel-release
sudo yum install redis-stack-server
sudo systemctl disable redis-stack-server.service

    Note: in order to start the redis-stack-server service, you may need to install the libssl module (e.g. libssl.so.10):

    sudo yum compat-openssl10.x86-64

  4. Create and edit the /etc/rc.local, add the following content to the file:

    #!/bin/sh -e

echo never > /sys/kernel/mm/transparent_hugepage/enabled
sysctl -w net.core.somaxconn=65535

exit 0

  5. Give executable permissions to the /etc/rc.local file by running the following command:

    sudo chmod +x /etc/rc.local

  6. Edit the /etc/sysctl.conf file, add the following line at the end of the file:

    vm.overcommit_memory=1

  7. Set ulimit values to ensures optimal compatibility and performance for Redis. Create the file /etc/security/limits.d/90-redis.conf with the following content:

    @redis - nofile 20480
@redis - stack  10240
@redis - nproc  10240

  8. Create the necessary folders by running the following commands:

    sudo mkdir -p /etc/redis/cluster/{7000,7001}
sudo mkdir -p /var/lib/redis/{7000,7001}
sudo mkdir -p /var/{run,log}/redis

  9. Create the file /etc/redis/cluster/7000/redis_7000.conf with the following content:

    protected-mode no
port 7000
bind 0.0.0.0

loadmodule /opt/redis-stack/lib/rediscompat.so
loadmodule /opt/redis-stack/lib/redisearch.so
loadmodule /opt/redis-stack/lib/redistimeseries.so
loadmodule /opt/redis-stack/lib/rejson.so
loadmodule /opt/redis-stack/lib/redisbloom.so
loadmodule /opt/redis-stack/lib/redisgears.so v8-plugin-path /opt/redis-stack/lib/libredisgears_v8_plugin.so

dir /var/lib/redis/7000/
appendonly yes

cluster-enabled yes
cluster-node-timeout 5000
cluster-config-file /etc/redis/cluster/7000/nodes_7000.conf

pidfile /var/run/redis/redis_7000.pid
logfile /var/log/redis/redis_7000.log
loglevel notice

requirepass [ACCESSKEY]
masterauth [ACCESSKEY]

enable-debug-command local

  10. Create the file /etc/redis/cluster/7001/redis_7001.conf with the following content:

    protected-mode no
port 7001
bind 0.0.0.0

loadmodule /opt/redis-stack/lib/rediscompat.so
loadmodule /opt/redis-stack/lib/redisearch.so
loadmodule /opt/redis-stack/lib/redistimeseries.so
loadmodule /opt/redis-stack/lib/rejson.so
loadmodule /opt/redis-stack/lib/redisbloom.so
loadmodule /opt/redis-stack/lib/redisgears.so v8-plugin-path /opt/redis-stack/lib/libredisgears_v8_plugin.so

dir /var/lib/redis/7001/
appendonly yes

cluster-enabled yes
cluster-node-timeout 5000
cluster-config-file /etc/redis/cluster/7001/nodes_7001.conf

pidfile /var/run/redis/redis_7001.pid
logfile /var/log/redis/redis_7001.log
loglevel notice

requirepass [ACCESSKEY]
masterauth [ACCESSKEY]

enable-debug-command local

  11. Create a redis user and a redis group for the Redis services and give them the correct permissions by running the following commands:

    sudo chown redis:redis -R /var/lib/redis
sudo chmod 770 -R /var/lib/redis
sudo chown redis:redis -R /etc/redis
sudo chown redis:redis /var/{run,log}/redis

  12. Create the file /etc/systemd/system/redis_7000.service with the following content:

    [Unit]
Description=Redis key-value database on 7000
After=network.target

[Service]
ExecStart=/opt/redis-stack/bin/redis-server /etc/redis/cluster/7000/redis_7000.conf --daemonize no --supervised no
Type=simple
User=redis
Group=redis
RuntimeDirectory=redis
RuntimeDirectoryMode=0755

[Install]
WantedBy=multi-user.target

  13. Create the file /etc/systemd/system/redis_7001.service with the following content:

    [Unit]
Description=Redis key-value database on 7001
After=network.target

[Service]
ExecStart=/usr/bin/redis-server /etc/redis/cluster/7001/redis_7001.conf --daemonize no --supervised no
Type=simple
User=redis
Group=redis
RuntimeDirectory=redis
RuntimeDirectoryMode=0755

[Install]
WantedBy=multi-user.target

  14. Tell systemd to start the two services redis_7000.service and redis_7001.service automatically at server boot by running the following commands:

    sudo systemctl enable /etc/systemd/system/redis_7000.service
sudo systemctl enable /etc/systemd/system/redis_7001.service

  15. Reboot the server:

    sudo reboot



Validate the installation of the Redis services

  1. Check the status of the redis_7000 and redis_7001 services by running the following command:

    $ sudo systemctl status redis_7000
● redis_7000.service - Redis key-value database on 7000
 Loaded: loaded (/etc/systemd/system/redis_7000.service; enabled; vendor preset: disabled)
 Active: active (running) since Thu 2025-03-27 03:42:15 UTC; 22min ago
Main PID: 727 (redis-server)
  Tasks: 7 (limit: 4650)
 Memory: 18.9M
    CPU: 3.365s
 CGroup: /system.slice/redis_7000.service
         └─ 727 "/opt/redis-stack/bin/redis-server *:7000 [cluster]"

  2. Check the redis_7000 and redis_7001 logs on all servers:

    $ tail -n 3 /var/log/redis/redis_7000.log
727:M 27 Mar 2025 03:42:17.641 * DB loaded from append only file: 0.312 seconds
727:M 27 Mar 2025 03:42:17.641 * Opening AOF incr file appendonly.aof.1.incr.aof on server start
727:M 27 Mar 2025 03:42:17.642 * Ready to accept connections tcp

You shouldn’t have any warnings in the logs files.

After checking that all servers are well configured, you can proceed with the Redis cluster configuration.



Set up the Redis Cluster



Configure the cluster

  1. Connect to one of the Redis server.

  2. To create a cluster run the following command:

    $ redis-cli --cluster create \
    10.0.0.124:7000 10.0.0.125:7000 10.0.0.126:7000 \
    10.0.0.124:7001 10.0.0.125:7001 10.0.0.126:7001 \
    --cluster-replicas 1 -a [ACCESSKEY]

    The output should be similar to the following:

    >>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 10.0.0.125:7001 to 10.0.0.124:7000
Adding replica 10.0.0.126:7001 to 10.0.0.125:7000
Adding replica 10.0.0.124:7001 to 10.0.0.126:7000
M: 41b58d5cea81103d296ee70e31aa56dfa0f1aa30 10.0.0.124:7000
   slots:[0-5460] (5461 slots) master
M: 2ae13d480218fc91235bdc727c57edef941817a7 10.0.0.125:7000
   slots:[5461-10922] (5462 slots) master
M: 0848285de9827407af8ee8da81bcc645be57793c 10.0.0.126:7000
   slots:[10923-16383] (5461 slots) master
S: 779283793850c6c8a30425ae2ef780114585a64e 10.0.0.124:7001
   replicates 0848285de9827407af8ee8da81bcc645be57793c
S: b2fa3c3902639db13e0a245a4f08f8657a4d06ac 10.0.0.125:7001
   replicates 41b58d5cea81103d296ee70e31aa56dfa0f1aa30
S: 095098c7e15b2e1021e215abe0ace4fd0f841328 10.0.0.126:7001
   replicates 2ae13d480218fc91235bdc727c57edef941817a7
Can I set the above configuration? (type 'yes' to accept):

  3. Type yes and press Enter to accept the proposed configuration.
    You get the configuration details in the output:

    Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 10.0.0.124:7000)
M: 41b58d5cea81103d296ee70e31aa56dfa0f1aa30 10.0.0.124:7000
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: 2ae13d480218fc91235bdc727c57edef941817a7 10.0.0.125:7000
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: 095098c7e15b2e1021e215abe0ace4fd0f841328 10.0.0.126:7001
slots: (0 slots) slave
replicates 2ae13d480218fc91235bdc727c57edef941817a7
S: 779283793850c6c8a30425ae2ef780114585a64e 10.0.0.124:7001
slots: (0 slots) slave
replicates 0848285de9827407af8ee8da81bcc645be57793c
S: b2fa3c3902639db13e0a245a4f08f8657a4d06ac 10.0.0.125:7001
slots: (0 slots) slave
replicates 41b58d5cea81103d296ee70e31aa56dfa0f1aa30
M: 0848285de9827407af8ee8da81bcc645be57793c 10.0.0.126:7000
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.



Checking the Redis Cluster

  1. Run the following command:

    redis-cli -c -h 10.0.0.124 -p 7000 -a [ACCESSKEY] cluster nodes

    The output should be similar to the following:

    2ae13d480218fc91235bdc727c57edef941817a7 10.0.0.125:7000@17000 master - 0 1743042394000 2 connected 5461-10922
095098c7e15b2e1021e215abe0ace4fd0f841328 10.0.0.126:7001@17001 slave 2ae13d480218fc91235bdc727c57edef941817a7 0 1743042394892 2 connected
779283793850c6c8a30425ae2ef780114585a64e 10.0.0.124:7001@17001 slave 0848285de9827407af8ee8da81bcc645be57793c 0 1743042394587 3 connected
b2fa3c3902639db13e0a245a4f08f8657a4d06ac 10.0.0.125:7001@17001 slave 41b58d5cea81103d296ee70e31aa56dfa0f1aa30 0 1743042394586 1 connected
41b58d5cea81103d296ee70e31aa56dfa0f1aa30 10.0.0.124:7000@17000 myself,master - 0 0 1 connected 0-5460
0848285de9827407af8ee8da81bcc645be57793c 10.0.0.126:7000@17000 master - 0 1743042393568 3 connected 10923-16383

    Since everything looks OK, perform a several SET and GET commands to test if the Redis Cluster behaves as expected.

  2. Connect to Redis Master:

    redis-cli -c -h 10.0.0.124 -p 7000 -a [ACCESSKEY]

  3. Run a several SET and GET commands to check the behavior of the Redis Cluster:

    10.0.0.124:7000> SET foo bar
-> Redirected to slot [12182] located at 10.0.0.126:7000
OK
10.0.0.124:7000> GET foo
"bar"

    As you can see the Redis sends the key to the correct hash slot.



Testing the failover

This section shows how to test the failover behavior of a Redis Cluster.



Master node failure

  1. Run the following command on Master node:

    redis-cli -c -h 127.0.0.0 -p 7000 -a [ACCESSKEY] DEBUG sleep 40

    This command makes this Master unreachable by the replicas during 40 seconds, forcing the associated Replica (port 7001) on server redis-2 to take over and change its role to Master.

  2. Check roles:

    $ redis-cli -c -h 127.0.0.0 -p 7000 -a [ACCESSKEY] info replication
# Replication
role:slave
master_host:10.0.0.125
master_port:7001
...

$ redis-cli -c -h 10.0.0.125 -p 7000 -a [ACCESSKEY] info replication
# Replication
role:master
connected_slaves:1
slave0:ip=10.0.0.126,port=7001,state=online,offset=3410,lag=1

$ redis-cli -c -h 10.0.0.124 -p 7000 -a [ACCESSKEY] cluster nodes
41b58d5cea81103d296ee70e31aa56dfa0f1aa30 10.0.0.124:7000@17000 myself,slave b2fa3c3902639db13e0a245a4f08f8657a4d06ac 0 0 7 connected
2ae13d480218fc91235bdc727c57edef941817a7 10.0.0.125:7000@17000 master - 0 1743045119565 2 connected 5461-10922
0848285de9827407af8ee8da81bcc645be57793c 10.0.0.126:7000@17000 master - 0 1743045118000 3 connected 10923-16383
779283793850c6c8a30425ae2ef780114585a64e 10.0.0.124:7001@17001 slave 0848285de9827407af8ee8da81bcc645be57793c 0 1743045119564 3 connected
b2fa3c3902639db13e0a245a4f08f8657a4d06ac 10.0.0.125:7001@17001 master - 0 1743045119053 7 connected 0-5460
095098c7e15b2e1021e215abe0ace4fd0f841328 10.0.0.126:7001@17001 slave 2ae13d480218fc91235bdc727c57edef941817a7 0 1743045118035 2 connected

    From the output, you can see that the Replica running in redis-2 got promoted to Master.



Replica node failure

  1. Run the following command on Replica in server redis-3:

    redis-cli -c -h 127.0.0.0 -p 7001 -a [ACCESSKEY] DEBUG sleep 40

  2. Check roles in the cluster:

    redis-cli -c -h 10.0.0.126 -p 7000 -a [ACCESSKEY] cluster nodes
41b58d5cea81103d296ee70e31aa56dfa0f1aa30 10.0.0.124:7000@17000 slave b2fa3c3902639db13e0a245a4f08f8657a4d06ac 0 1743046096000 7 connected
779283793850c6c8a30425ae2ef780114585a64e 10.0.0.124:7001@17001 slave 0848285de9827407af8ee8da81bcc645be57793c 0 1743046097459 3 connected
2ae13d480218fc91235bdc727c57edef941817a7 10.0.0.125:7000@17000 master - 0 1743046097900 2 connected 5461-10922
b2fa3c3902639db13e0a245a4f08f8657a4d06ac 10.0.0.125:7001@17001 master - 0 1743046096395 7 connected 0-5460
0848285de9827407af8ee8da81bcc645be57793c 10.0.0.126:7000@17000 myself,master - 0 0 3 connected 10923-16383
095098c7e15b2e1021e215abe0ace4fd0f841328 10.0.0.126:7001@17001 slave 2ae13d480218fc91235bdc727c57edef941817a7 0 1743046095519 2 connected

    As you can see, the cluster topology isn’t changed.

Congrats! You have configured the Redis Cluster.



Summary

In this post, we have seen:

  • how to install the Redis Stack Server
  • how to set up the Redis Cluster
  • hot to test the failover of a cluster.



Source link

Leave a Reply

Your email address will not be published. Required fields are marked *

Related News