1 At least 6 nodes are required(3 masters and 3 slaves):
if use followed configure
# Settings
PORT=30000
TIMEOUT=2000
NODES=4 //all nodes number include master
REPLICAS=1
it will popup error:
[root@wbxperf001 create-cluster]# ./create-cluster start
Starting 30001
Starting 30002
Starting 30003
Starting 30004
[root@wbxperf001 create-cluster]# ./create-cluster create
*** ERROR: Invalid configuration for cluster creation.
*** Redis Cluster requires at least 3 master nodes.
*** This is not possible with 4 nodes and 1 replicas per node.
*** At least 6 nodes are required.
root cause:
redis-trib.rb
def check_create_parameters
masters = @nodes.length/(@replicas+1)
if masters < 3
puts "*** ERROR: Invalid configuration for cluster creation."
puts "*** Redis Cluster requires at least 3 master nodes."
puts "*** This is not possible with #{@nodes.length} nodes and #{@replicas} replicas per node."
puts "*** At least #{3*(@replicas+1)} nodes are required."
exit 1
end
end
2 pkill master, slave will on, then start the old master, it will be slave
[root@wbxperf001 ~]# cluster nodes
1fbaae0a5e98f2f4bb299139ffd126811d68fbbe 127.0.0.1:<span style="color: #0000ff;">30005</span> <span style="color: #0000ff;">slave</span> b067d0613418238a688f18ebd6a8e3612c1bb54b 0 1496327104008 5 connected
d8fd8928d10e11c9eddb0e80f40a5345f092d40c 127.0.0.1:30004 slave 7cf0fd6e2dce793161f0764ff7ef6e7f83b54c05 0 1496327104008 4 connected
c5a97527cdaa550d89e7282c0e888ea3b0d53a29 127.0.0.1:30006 slave b934ab47ef37a374498067f864a067a36674debc 0 1496327103909 6 connected
b934ab47ef37a374498067f864a067a36674debc 127.0.0.1:30003 myself,master - 0 0 3 connected 10923-16383
7cf0fd6e2dce793161f0764ff7ef6e7f83b54c05 127.0.0.1:30001 master - 0 1496327104009 1 connected 0-5460
b067d0613418238a688f18ebd6a8e3612c1bb54b 127.0.0.1:<span style="color: #ff0000;">30002</span> <span style="color: #ff0000;">master</span> - 0 1496327104008 2 connected 5461-10922
[root@wbxperf001 ~]# ps -ef|grep redis
root 9590 1 0 06:22 ? 00:01:33 ../../src/redis-server *:30001 [cluster]
root <span style="color: #ff0000;">9592</span> 1 0 06:22 ? 00:01:31 ../../src/redis-server *:<span style="color: #ff0000;">30002</span> [cluster]
root 9598 1 0 06:22 ? 00:01:32 ../../src/redis-server *:30003 [cluster]
root 9602 1 0 06:22 ? 00:01:32 ../../src/redis-server *:30004 [cluster]
root 9606 1 0 06:22 ? 00:01:27 ../../src/redis-server *:30005 [cluster]
root 9610 1 0 06:22 ? 00:01:31 ../../src/redis-server *:30006 [cluster]
[root@wbxperf001 ~]# kill -1 <span style="color: #ff0000;">9592</span>
[root@wbxperf001 ~]# cluster nodes
1fbaae0a5e98f2f4bb299139ffd126811d68fbbe 127.0.0.1:<span style="color: #0000ff;">30005 master </span>- 0 1496327180237 7 connected 5461-10922
d8fd8928d10e11c9eddb0e80f40a5345f092d40c 127.0.0.1:30004 slave 7cf0fd6e2dce793161f0764ff7ef6e7f83b54c05 0 1496327180237 4 connected
c5a97527cdaa550d89e7282c0e888ea3b0d53a29 127.0.0.1:30006 slave b934ab47ef37a374498067f864a067a36674debc 0 1496327180237 6 connected
b934ab47ef37a374498067f864a067a36674debc 127.0.0.1:30003 myself,master - 0 0 3 connected 10923-16383
7cf0fd6e2dce793161f0764ff7ef6e7f83b54c05 127.0.0.1:30001 master - 0 1496327180237 1 connected 0-5460
b067d0613418238a688f18ebd6a8e3612c1bb54b 127.0.0.1:<span style="color: #ff0000;">30002</span> <span style="color: #ff0000;">master</span>,<span style="color: #ff0000;">fail</span> - 1496327177426 1496327177227 2 disconnected
//after startup 30002, it will always be slave.
so if want to set it as master instead of keep slave, use cluster failover:
[root@wbxperf001 src]# ./redis-cli -p 30002
127.0.0.1:30002> cluster failover
127.0.0.1:30002> cluster nodes
c5a97527cdaa550d89e7282c0e888ea3b0d53a29 127.0.0.1:30006 slave b934ab47ef37a374498067f864a067a36674debc 0 1496328987578 6 connected
b934ab47ef37a374498067f864a067a36674debc 127.0.0.1:30003 master - 0 1496328987578 3 connected 10923-16383
7cf0fd6e2dce793161f0764ff7ef6e7f83b54c05 127.0.0.1:30001 master - 0 1496328987578 1 connected 0-5460
b067d0613418238a688f18ebd6a8e3612c1bb54b 127.0.0.1:30002 myself,master - 0 0 8 connected 5461-10922
d8fd8928d10e11c9eddb0e80f40a5345f092d40c 127.0.0.1:30004 slave 7cf0fd6e2dce793161f0764ff7ef6e7f83b54c05 0 1496328987578 4 connected
1fbaae0a5e98f2f4bb299139ffd126811d68fbbe 127.0.0.1:30005 slave b067d0613418238a688f18ebd6a8e3612c1bb54b 0 1496328987578 8 connected
3 if one of masters down, if cluster down dependent on parameter: cluster-require-full-coverage
cluster-require-full-coverage <yes/no>: If this is set to yes, as it is by default, the cluster stops accepting writes if some percentage of the key space is not covered by any node. If the option is set to no, the cluster will still serve queries even if only requests about a subset of keys can be processed.
4 Cluster doesn’t support ?
(1) multi-key operation.
(2) select db, just use db 0:
Redis Cluster does not support multiple databases like the stand alone version of Redis. There is just database 0 and the SELECT command is not allowed.
5 some command can’t run any nodes:
127.0.0.1:7001> cluster failover
(error) ERR You should send CLUSTER FAILOVER to a slave
6 is redis-benchmark support cluster test?
no. but if using default parameter with -e. It won’t popup any error, so you may believe it can support cluster’s test
" -e If server replies with errors, show them on stdout.\n"
" (no more than 1 error per second is displayed)\n"
it is interesting for the error number control.
if (config.showerrors) {
static time_t lasterr_time = 0;
time_t now = time(NULL);
redisReply *r = reply;
if (r->type == REDIS_REPLY_ERROR && lasterr_time != now) {
lasterr_time = now;
printf("Error from server: %s\n", r->str);
}
}
7 cluster上任意节点执行keys *
返回的是当前节点负责的数据区域（slots）,而不是整个cluster的数据。
另外set key value时，如果当前的数据不应该由自己负责存储，则返回moved error异常：
127.0.0.1:7001> keys *
1) "fujian1234"
127.0.0.1:7001>
127.0.0.1:7001>
127.0.0.1:7001> set fujian1234 value
(error) MOVED 15336 10.224.2.144:7001
加-c后可以自动redirect：
The redis-cli utility in the unstable branch of the Redis repository at GitHub implements a very basic cluster support when started with the -c switch.
[root@redis001 ~]# redis-cli -p 7001 -a P@ss123 -c
127.0.0.1:7001> get fujian1234
-> Redirected to slot [15336] located at 10.224.2.144:7001
"value"
此外，在cluster slave上执行读请求，即使数据在所在的数据区域，也可能moved error.同样可以加-c,这种情况下，也可以执行readonly。
slave moved error:
[root@redis004 bin]# ./redis-cli -p 7001 -a P@ss123
127.0.0.1:7001> get fujian1234
(error) MOVED 15336 10.224.2.141:7001
127.0.0.1:7001> readonly
OK
127.0.0.1:7001> get fujian1234
"xinxiu"
Read queries against a Redis Cluster slave node are disabled by default, but you can use the READONLY command to change this behavior on a per- connection basis. The READWRITE command resets the readonly mode flag of a connection back to readwrite.
READWRITE Disables read queries for a connection to a Redis Cluster slave node.
所以引发另外一个问题的思考： 配置里面的slave_read-only和READONLY的关系和区别：
答案引用一段话：
Please take note that slave-read-only config refers to replication and READONLY refers to the redis-cluster command.
If you are not using redis-cluster, you can safely ignore the READONLY command documentation. Refer to https://raw.githubusercontent.com/antirez/redis/2.8/redis.conf instead. Writes should not replicate nor require lookups to the master. My wireshark dumps on redis with slave-read-only no shows no indication of any communication with master as a consequence of writes to the slave itself.
If you are using redis-cluster on the other hand, and referring to the READWRITE behavior: Cluster nodes' communication with each other for hash slot updates and other cluster specific messages are optimized to use minimal bandwidth and the least processing time. Communicating hash slot updates most likely do not happen for every write on the slave.
slave-read-only实际测试对redis cluster而言并无作用。
