RocketMQ Dledger 模式集群搭建

  • Post author:
  • Post category:其他


Deledger 集群模式是多副本模式,每个Master配置二个 Slave 组成 Dledger Group,可以有多个 Dledger Group,由 Dledger 实现 Master 选举,具体定义的时候核心在于ID。

考虑到可维护性的特点,本次Broker集群采用的是DLeger多副本模式,这样就需要用过Broker配置文件的方式定义每一个DLegerGroup,再每一个DLegerGroup 中必须存在有三个Broker节点,这样就需要为每一个节点设置一个ID标记,并且通过此标记来绑定与之相关的主机地址。

再配置DLegerID的时候有一个重要的命名要求:第一个随意编写,第2个开始必须为数字,这一点如果没有正确配置,则再启动的时候就会出现错误,而这一设置要求是有聚义的源代码来定义的。


本次集群搭建所用的主机

主机名 IP 功能

BROKER 角色

nameserver-cluster-a

172.17.10.50

NameServer+控制台

nameserver-cluster-b

172.17.10.51

NameServer+控制台

broker-cluster-a-node-1

172.17.10.52

Broker

集群决策

broker-cluster-a-node-2

172.17.10.53

Broker

集群决策

broker-cluster-a-node-3

172.17.10.54

Broker

集群决策

broker-cluster-b-node-1

172.17.10.55

Broker

集群决策

broker-cluster-b-node-2

172.17.10.56

Broker

集群决策

broker-cluster-b-node-3

172.17.10.57

Broker

集群决策

broker-cluster-c-node-1 172.17.10.58

Broker

集群决策

broker-cluster-c-node-2 172.17.10.59

Broker

集群决策

broker-cluster-c-node-3 172.17.10.60

Broker

集群决策

所有节点安装基础环境:


以单个节点举例:

[root@nameserver-cluster-a ~]# yum -y install wget unzip  java-1.8.0-openjdk*
[root@nameserver-cluster-a ~]# java -version
openjdk version "1.8.0_322"
OpenJDK Runtime Environment (build 1.8.0_322-b06)
OpenJDK 64-Bit Server VM (build 25.322-b06, mixed mode)

#下载rocketmq二进制包
[root@nameserver-cluster-a ~]# wget https://archive.apache.org/dist/rocketmq/4.8.0/rocketmq-all-4.8.0-bin-release.zip


添加环境变量:


再/etc/profile 下添加

export JAVA_HOME=/usr/lib/jvm/java
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/jre/lib/rt.jar
export PATH=$PATH:$JAVA_HOME/bin

source /etc/profile  生效


【可选】添加hosts

172.17.10.50	nameserver-cluster-a
172.17.10.51	nameserver-cluster-b
172.17.10.52	broker-cluster-a-node-1
172.17.10.53	broker-cluster-a-node-2
172.17.10.54	broker-cluster-a-node-3
172.17.10.55	broker-cluster-b-node-1
172.17.10.56	broker-cluster-b-node-2
172.17.10.57	broker-cluster-b-node-3
172.17.10.58	broker-cluster-b-node-1
172.17.10.59	broker-cluster-b-node-2
172.17.10.60	broker-cluster-b-node-3

安装Nameserver集群:

NameServer 是一个重复的数据存储集群,每一个NameServer 保存的数据完成相同,并且没有所谓的数据同步的处理操作,那么再进行服务搭建的时候每个服务同时配置相同的内容即可。

NameServer保存了整了 RocketMQ 集群之中的所有核心数据 (主题数据、Broker 数据),生产者和消费者都需要通过 NameServer 主机过去相关的 RocketMQ 的信息。


基本步骤

【1】 再nameserver-cluster-a主机上进行安装配置

【2】 将配置好的rocketmq 复制给nameserver-cluster-b

【3】 启动服务,并开放防火墙端口


安装Namesvc


1、解压到指定目录并更改名称:

[root@nameserver-cluster-a ~]# unzip rocketmq-all-4.8.0-bin-release.zip  -d /opt/
[root@nameserver-cluster-a ~]# cd /opt/
[root@nameserver-cluster-a opt]# ls
rocketmq  rocketmq-all-4.8.0-bin-release
[root@nameserver-cluster-a opt]# mv rocketmq-all-4.8.0-bin-release/ rocketmq


2、建立日志目录

#日志目录若不是再rockermq二进制包路径内,需要再其他namesvc节点也执行下
[root@nameserver-cluster-a ~]#  mkdir -p /opt/rocketmq/logs


3、配置JVM内存并更改日志存储路径

[root@nameserver-cluster-a opt]# vim rocketmq/bin/runserver.sh
#根据自身情况去更改此默认值
JAVA_OPT="${JAVA_OPT} -server -Xms4g -Xmx4g -Xmn2g -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"
choose_gc_options
JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow"
JAVA_OPT="${JAVA_OPT} -XX:-UseLargePages"
JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${JAVA_HOME}/jre/lib/ext:${BASE_DIR}/lib:${JAVA_HOME}/lib/ext"
#rewrite user home 添加行(再Djava.ext.dirs行下添加)
JAVA_OPT="${JAVA_OPT} -Duser.home=/opt/rocketmq/logs"


4、将配置完的namesvc发送到其他namesvc主机

[root@nameserver-cluster-a opt]# scp -r rocketmq/  nameserver-cluster-b:/opt/


5、所有NameServer节点启动

[root@nameserver-cluster-a opt]# nohup  /opt/rocketmq/bin/mqnamesrv > /opt/rocketmq/logs/rocketmq-namesrv.log 2>&1 &
[1] 10742
[root@nameserver-cluster-a opt]# jps 
10761 NamesrvStartup
10779 Jps
[root@nameserver-cluster-a opt]# netstat  -anptu |grep 9876
tcp6       0      0 :::9876                 :::*                    LISTEN      10761/java


[root@nameserver-cluster-b opt]# nohup  /opt/rocketmq/bin/mqnamesrv > /opt/rocketmq/logs/rocketmq-namesrv.log 2>&1 &
[1] 14033
[root@nameserver-cluster-b opt]# jps 
14052 NamesrvStartup
14070 Jps
[root@nameserver-cluster-b opt]# netstat  -anptu|grep 9876
tcp6       0      0 :::9876                 :::*                    LISTEN      14052/java 


6、防火墙开放此端口

firewall-cmd  --zone=public --add-port=9876/tcp --permanent
firewall-cmd --reload


7、RocketMQ nameserver节点关闭命令

/opt/rocketmq/bin/mqshutdown  namesrv

安装Broker集群

是整个 BocketMQ 集群之中的核心所在,所以消息的传输以及存储的处理全部有该集群负责完成,所有的 Broker 启动之后都会自动的向所有的 Nameserver 发送注册信息(路由信息)


基本步骤


再broker单台主机上安装,后同步到其他broker主机后进行修改

【1】 再broker-cluster-a-node-1 进行集群配置

【2】 将broker-cluster-a-node-1 上进行的配置同步给broker-cluster-a-node-*主机上

【3】 修改 broker-cluster-* 节点上的配置进行修改


再所有Broker节点创建日志、数据存储目录

 mkdir -p /opt/rocketmq/{data/{store,commitlog},logs}

安装配置一个broker节点


1、解压重命名RocketMQ二进制包


[root@broker-cluster-a-node-1 ~]# unzip rocketmq-all-4.8.0-bin-release.zip  -d /opt/
[root@broker-cluster-a-node-1 ~]# cd /opt/
[root@broker-cluster-a-node-1 opt]# mv rocketmq-all-4.8.0-bin-release/ rocketmq


2、修改RocketMQ  Dledger模式配置

[root@broker-cluster-a-node-1 opt]# cp  /opt/rocketmq/conf/dledger/broker-n0.conf  /opt/rocketmq/conf/dledger/broker-cluster.conf
[root@broker-cluster-a-node-1 opt]# vim /opt/rocketmq/conf/dledger/broker-cluster.conf
#定义集群的名称
brokerClusterName = PrivacyCluster  

#定义Broker的名称
brokerName=broker-PrivacyCluster-A

#服务监听端口
listenPort=30911

#nameserver节点
namesrvAddr=172.17.10.50:9876;172.17.10.51:9876

#数据存储目录(消息存储路径)
storePathRootDir=/opt/rocketmq/data/store

#commitlog目录
storePathCommitLog=/opt/rocketmq/data/commitlog/

#是否使用DLeger 模式
enableDLegerCommitLog=true

#DLeger分组,一般与brokerName相同
dLegerGroup=broker-PrivacyCluster-A

#集群中其他节点,实现Dleger内部通信(命名规则,第一个字母为自定义,从第二个开始必须为数字 (n10是ID)
dLegerPeers=n10-172.17.10.52:40911;n11-172.17.10.53:40911;n12-172.17.10.54:40911
## must be unique

#当前节点信息,上面定义的ID
dLegerSelfId=n10

#发送线程池的大小,建议配置成CPU核数
sendMessageThreadPoolNums=16

#========追加配置项===========
#开启异步刷盘
flushDiskType=ASYNC_FLUSH

#线上关闭自动创建topic
autoCreateTopicEnable=false

#开启临时存储池-异步刷盘建议开启
TransientStorePoolEnable=true

#同步刷盘建议使用重入锁-异步建议关闭
useReentrantLockWhenPutMessage=false

#发送消息的最大线程数,默认1,同步刷盘建议适当增大,建议配置成CPU核数
sendMessageThreadPoolNums=8

#关闭堆内存数据传输-建议关闭,可以提高拉消息效率,(false可能导致消费者超时 TPS上不去)
#transferMsgByHeap=false
transferMsgByHeap=true

#开启消息轨迹,建议再集群中新增加个节点,并仅再新增节点上开启消息轨迹
#traceTopicEnable=true

#开启从Slave读数据功能
slaveReadEnable=true

#linux建议开启 Epoll IO模型
useEpollNativeSelector=true

#启用ACL访问控制(不添加ACL访问控制,可直接跳过ACL访问控制的配置)
aclEnable=true


3、配置Broker JVM 内存以及日志目录

[root@broker-cluster-a-node-1 opt]# vim /opt/rocketmq/bin/runbroker.sh
#根据自身情况更改内存配置
JAVA_OPT="${JAVA_OPT} -server -Xms8g -Xmx8g -Xmn4g"
JAVA_OPT="${JAVA_OPT} -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 -XX:SoftRefLRUPolicyMSPerMB=0"
JAVA_OPT="${JAVA_OPT} -verbose:gc -Xloggc:${GC_LOG_DIR}/rmq_broker_gc_%p_%t.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintAdaptiveSizePolicy"
JAVA_OPT="${JAVA_OPT} -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m"
JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow"
JAVA_OPT="${JAVA_OPT} -XX:+AlwaysPreTouch"
JAVA_OPT="${JAVA_OPT} -XX:MaxDirectMemorySize=15g"
JAVA_OPT="${JAVA_OPT} -XX:-UseLargePages -XX:-UseBiasedLocking"
JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${JAVA_HOME}/jre/lib/ext:${BASE_DIR}/lib:${JAVA_HOME}/lib/ext"
#新增日志路径配置
#rewrite user home 
JAVA_OPT="${JAVA_OPT} -Duser.home=/opt/rocketmq/logs"


4、配置ACL访问控制


aclEnable=false 或者未添加aclEnable配置可忽略这项配置

[root@broker-cluster-a-node-1 opt]# vim /opt/rocketmq/conf/plain_acl.yml
#定义IP白名单
globalWhiteRemoteAddresses:
- 10.10.103.*
- 192.168.*.*
- 172.17.10.*

#应以用户信息以及对注意的权限
accounts:

  #用户名
- accessKey:  指定允许的用户名

  #密码
  secretKey:  指定允许的用户名的密码

  #用户级别的 IP 地址白名单配置,可根据需求自行修改
  whiteRemoteAddress:

  ##该配置项为是否设置管理员,
  admin: false

  #默认Topic主题: 拒绝
  defaultTopicPerm: DENY

  #默认主题组SUB权限
  defaultGroupPerm: SUB

  #配置该用户对主题的权限
  topicPerms:

  #对topicPrivacy主要发布订阅  
  - topicPrivacy=PUB|SUB
  #对Topictest主要发布订阅 
  - Topictest=PUB|SUB
  groupPerms:
  # the group should convert to retry topic
  - topicPrivacy=PUB|SUB
  - Topictest=PUB|SUB

- accessKey: rocketmqadmin
  secretKey: y2Ff!Ds46jYSki^Da
  whiteRemoteAddress: 192.168.1.*,172.17.10.*
  # if it is admin, it could access all resources
  admin: true


配置参考:

配置其他的Broker节点


1、将配置好的RocketMQ 目录传输给其他broker主机

[root@broker-cluster-a-node-1 opt]# scp -r rocketmq/ broker-cluster-a-node-2:/opt/
[root@broker-cluster-a-node-1 opt]# scp -r rocketmq/ broker-cluster-a-node-3:/opt/
[root@broker-cluster-a-node-1 opt]# scp -r rocketmq/ broker-cluster-b-node-1:/opt/
[root@broker-cluster-a-node-1 opt]# scp -r rocketmq/ broker-cluster-b-node-2:/opt/
[root@broker-cluster-a-node-1 opt]# scp -r rocketmq/ broker-cluster-b-node-3:/opt/
[root@broker-cluster-a-node-1 opt]# scp -r rocketmq/ broker-cluster-c-node-1:/opt/
[root@broker-cluster-a-node-1 opt]# scp -r rocketmq/ broker-cluster-c-node-2:/opt/
[root@broker-cluster-a-node-1 opt]# scp -r rocketmq/ broker-cluster-c-node-3:/opt/


2、修改broker-cluster-a-node-2主机的MQ配置

[root@broker-cluster-a-node-2 ~]# vim /opt/rocketmq/conf/dledger/broker-cluster.conf
brokerClusterName = PrivacyCluster
brokerName=broker-PrivacyCluster-A
listenPort=30911
namesrvAddr=172.17.10.50:9876;172.17.10.51:9876
storePathRootDir=/opt/rocketmq/data/store
storePathCommitLog=/opt/rocketmq/data/commitlog/
enableDLegerCommitLog=true
dLegerGroup=broker-PrivacyCluster-A
dLegerPeers=n10-172.17.10.52:40911;n11-172.17.10.53:40911;n12-172.17.10.54:40911
## must be unique
dLegerSelfId=n11  #修改此处
sendMessageThreadPoolNums=16
flushDiskType=ASYNC_FLUSH
autoCreateTopicEnable=false
TransientStorePoolEnable=true
useReentrantLockWhenPutMessage=false
sendMessageThreadPoolNums=8
#transferMsgByHeap=false
transferMsgByHeap=true
slaveReadEnable=true
useEpollNativeSelector=true


3、修改broker-cluster-a-node-3主机的MQ配置

[root@broker-cluster-a-node-3 ~]# vim  /opt/rocketmq/conf/dledger/broker-cluster.conf
brokerClusterName = PrivacyCluster
brokerName=broker-PrivacyCluster-A
listenPort=30911
namesrvAddr=172.17.10.50:9876;172.17.10.51:9876
storePathRootDir=/opt/rocketmq/data/store
storePathCommitLog=/opt/rocketmq/data/commitlog/
enableDLegerCommitLog=true
dLegerGroup=broker-PrivacyCluster-A
dLegerPeers=n10-172.17.10.52:40911;n11-172.17.10.53:40911;n12-172.17.10.54:40911
## must be unique 
dLegerSelfId=n12  #修改此处
sendMessageThreadPoolNums=16
flushDiskType=ASYNC_FLUSH
autoCreateTopicEnable=false
TransientStorePoolEnable=true
useReentrantLockWhenPutMessage=false
sendMessageThreadPoolNums=8
#transferMsgByHeap=false
transferMsgByHeap=true
slaveReadEnable=true
useEpollNativeSelector=true


4、启动 broker-cluster-a-*这组服broker

nohup /opt/rocketmq/bin/mqbroker -c /opt/rocketmq/conf/dledger/broker-cluster.conf   > /opt/rocketmq/logs/broker.log 2>&1 & 
[root@broker-cluster-a-node-1 ~]# jps 
8276 Jps
7515 BrokerStartup
[root@broker-cluster-a-node-2 ~]# jps 
11443 Jps
10663 BrokerStartup
[root@broker-cluster-a-node-3 ~]# jps 
11593 Jps
10827 BrokerStartup

/opt/rocketmq/bin/mqadmin   clusterList -n ” nameserver_ip:9876″   查看集群主机


可选择在这里现将控制台安装上:


5、 修改broker-cluster-b-noe-1主机的MQ配置

brokerClusterName = PrivacyCluster
brokerName=broker-PrivacyCluster-B #修改此处
listenPort=30911
namesrvAddr=172.17.10.50:9876;172.17.10.51:9876
storePathRootDir=/opt/rocketmq/data/store
storePathCommitLog=/opt/rocketmq/data/commitlog/
enableDLegerCommitLog=true
dLegerGroup=broker-PrivacyCluster-B #修改此处
dLegerPeers=n21-172.17.10.55:40911;n22-172.17.10.56:40911;n23-172.17.10.57:40911 #修改此处
## must be unique
dLegerSelfId=n21 #修改此处
sendMessageThreadPoolNums=16
flushDiskType=ASYNC_FLUSH
autoCreateTopicEnable=false
TransientStorePoolEnable=true
useReentrantLockWhenPutMessage=false
sendMessageThreadPoolNums=8
#transferMsgByHeap=false
transferMsgByHeap=true
slaveReadEnable=true
useEpollNativeSelector=true


6、 修改broker-cluster-b-node-2主机的MQ配置

brokerClusterName = PrivacyCluster
brokerName=broker-PrivacyCluster-B #修改此处
listenPort=30911
namesrvAddr=172.17.10.50:9876;172.17.10.51:9876
storePathRootDir=/opt/rocketmq/data/store
storePathCommitLog=/opt/rocketmq/data/commitlog/
enableDLegerCommitLog=true
dLegerGroup=broker-PrivacyCluster-B #修改此处
dLegerPeers=n21-172.17.10.55:40911;n22-172.17.10.56:40911;n23-172.17.10.57:40911 #修改此处
## must be unique
dLegerSelfId=n22 #修改此处
sendMessageThreadPoolNums=16
flushDiskType=ASYNC_FLUSH
autoCreateTopicEnable=false
TransientStorePoolEnable=true
useReentrantLockWhenPutMessage=false
sendMessageThreadPoolNums=8
#transferMsgByHeap=false
transferMsgByHeap=true
slaveReadEnable=true
useEpollNativeSelector=true


7、 修改broker-cluster-b-node-3主机的MQ配置

brokerClusterName = PrivacyCluster
brokerName=broker-PrivacyCluster-B #修改此处
listenPort=30911
namesrvAddr=172.17.10.50:9876;172.17.10.51:9876
storePathRootDir=/opt/rocketmq/data/store
storePathCommitLog=/opt/rocketmq/data/commitlog/
enableDLegerCommitLog=true
dLegerGroup=broker-PrivacyCluster-B #修改此处
dLegerPeers=n21-172.17.10.55:40911;n22-172.17.10.56:40911;n23-172.17.10.57:40911 #修改此处
## must be unique 
dLegerSelfId=n23 #修改此处
sendMessageThreadPoolNums=16
flushDiskType=ASYNC_FLUSH
autoCreateTopicEnable=false
TransientStorePoolEnable=true
useReentrantLockWhenPutMessage=false
sendMessageThreadPoolNums=8
#transferMsgByHeap=false
transferMsgByHeap=true
slaveReadEnable=true
useEpollNativeSelector=true


8、启动 broker-cluster-b-*这组broker

nohup /opt/rocketmq/bin/mqbroker -c /opt/rocketmq/conf/dledger/broker-cluster.conf   > /opt/rocketmq/logs/broker.log 2>&1 & 
[root@broker-cluster-b-node-1 ~]# jps 
11984 BrokerStartup
12105 Jps
[root@broker-cluster-b-node-2 ~]# jps 
11996 BrokerStartup
12156 Jps
[root@broker-cluster-b-node-3 ~]# jps 
13292 Jps
13165 BrokerStartup


9、修改broker-cluster-c-node-1主机的MQ配置

brokerClusterName = PrivacyCluster
brokerName=broker-PrivacyCluster-C  #修改此处
listenPort=30911
namesrvAddr=172.17.10.50:9876;172.17.10.51:9876
storePathRootDir=/opt/rocketmq/data/store
storePathCommitLog=/opt/rocketmq/data/commitlog/
enableDLegerCommitLog=true
dLegerGroup=broker-PrivacyCluster-C #修改此处
dLegerPeers=n31-172.17.10.58:40911;n32-172.17.10.59:40911;n33-172.17.10.60:40911 #修改此处
## must be unique
dLegerSelfId=n31  #修改此处
sendMessageThreadPoolNums=16
flushDiskType=ASYNC_FLUSH
autoCreateTopicEnable=false
TransientStorePoolEnable=true
useReentrantLockWhenPutMessage=false
sendMessageThreadPoolNums=8
#transferMsgByHeap=false
transferMsgByHeap=true
slaveReadEnable=true
useEpollNativeSelector=true


10、修改broker-cluster-c-node-2主机的MQ配置

brokerClusterName = PrivacyCluster
brokerName=broker-PrivacyCluster-C #修改此处
listenPort=30911 
namesrvAddr=172.17.10.50:9876;172.17.10.51:9876
storePathRootDir=/opt/rocketmq/data/store
storePathCommitLog=/opt/rocketmq/data/commitlog/
enableDLegerCommitLog=true
dLegerGroup=broker-PrivacyCluster-C #修改此处
dLegerPeers=n31-172.17.10.58:40911;n32-172.17.10.59:40911;n33-172.17.10.60:40911 #修改此处
## must be unique
dLegerSelfId=n32  #修改此处
sendMessageThreadPoolNums=16
flushDiskType=ASYNC_FLUSH
autoCreateTopicEnable=false
TransientStorePoolEnable=true
useReentrantLockWhenPutMessage=false
sendMessageThreadPoolNums=8
#transferMsgByHeap=false
transferMsgByHeap=true
slaveReadEnable=true
useEpollNativeSelector=true


11、修改broker-cluster-c-node-3主机的MQ配置

brokerClusterName = PrivacyCluster
brokerName=broker-PrivacyCluster-C  #修改此处
listenPort=30911
namesrvAddr=172.17.10.50:9876;172.17.10.51:9876
storePathRootDir=/opt/rocketmq/data/store
storePathCommitLog=/opt/rocketmq/data/commitlog/
enableDLegerCommitLog=true
dLegerGroup=broker-PrivacyCluster-C  #修改此处
dLegerPeers=n31-172.17.10.58:40911;n32-172.17.10.59:40911;n33-172.17.10.60:40911  #修改此处
## must be unique
dLegerSelfId=n33  #修改此处
sendMessageThreadPoolNums=16
flushDiskType=ASYNC_FLUSH
autoCreateTopicEnable=false
TransientStorePoolEnable=true
useReentrantLockWhenPutMessage=false
sendMessageThreadPoolNums=8
#transferMsgByHeap=false
transferMsgByHeap=true
slaveReadEnable=true
useEpollNativeSelector=true


12、启动 broker-cluster-c-*这组broker

nohup /opt/rocketmq/bin/mqbroker -c /opt/rocketmq/conf/dledger/broker-cluster.conf   > /opt/rocketmq/logs/broker.log 2>&1 & 
[root@broker-cluster-c-node-1 opt]# jps 
10962 Jps
10844 BrokerStartup

[root@broker-cluster-c-node-2 ~]# jps 
10529 BrokerStartup
10628 Jps

[root@broker-cluster-c-node-3 opt]# jps 
10917 BrokerStartup
11022 Jps


13、Broker 节点开放防火墙

firewall-cmd --zone=public --add-port=30909/tcp --permanent
firewall-cmd --zone=public --add-port=30911/tcp --permanent
firewall-cmd --zone=public --add-port=40911/tcp --permanent
firewall-cmd --reload


14、RocketMQ Broker节点关闭命令

/opt/rocketmq/bin/mqshutdown  broker

安装管理控制台

RocketMQ 实际上是属于一个“丐版”的消息组件,没有自身提供的管理控制台,如果需要使用控制台进行管理,则需要通过第三方的组件完成。


在Nameserver服务器上安装控制台


1、下载maven

[root@nameserver-cluster-a ~]# wget https://archive.apache.org/dist/maven/maven-3/3.8.4/binaries/apache-maven-3.8.4-bin.tar.gz
[root@nameserver-cluster-a ~]# tar xf apache-maven-3.8.4-bin.tar.gz
[root@nameserver-cluster-a ~]# unzip rocketmq-externals-master.zip  -d /opt/

2、

下载控制台:




GitHub – muyan-yootk/rocketmq-externals: Mirror of Apache RocketMQ (Incubating)



https://github.com/muyan-yootk/rocketmq-externals


3、

修改控制台配置文件:



修改配置文件

[root@nameserver-cluster-a ~]# vim /opt/rocketmq-externals-master/rocketmq-console/src/main/resources/application.properties
#Nameserver地址,多个已";"分割
rocketmq.config.namesrvAddr=172.17.10.50:9876;172.17.10.51:9876
#ACL中定义的用户
rocketmq.config.accessKey=admin
#ACL中定义的密码
rocketmq.config.secretKey=
#开启控制台登录权限认证
rocketmq.config.loginRequired=true

修改控制台用户配置文件

[root@nameserver-cluster-a ~]# vim /opt/rocketmq-externals-master/rocketmq-console/src/main/resources/users.properties
# Define Admin   #1:管理员  0:普通用户
admin=Acb123DBC@#123,1  

# Define Users
rocketmq1=rocketmq1!@#123
rocketmq2=rocketmq2!@#123


4、打包控制台代码

[root@nameserver-cluster-a rocketmq-console]# /root/apache-maven-3.8.4/bin/mvn  clean package -Dmaven.test.skip=true


5、运行控制台并登录

[root@nameserver-cluster-a rocketmq-console]# mv target/rocketmq-console-ng-1.0.1.jar /opt/rocketmq
[root@nameserver-cluster-a rocketmq-console]# java -jar /opt/rocketmq/rocketmq-console-ng-1.0.1.jar >/opt/rocketmq/logs/rocketmq-console.log  2>&1 &


账户:admin


密码:Acb123DBC@#123


创建一个主题:

RocketMQ自带测试:

1、配置环境变量:

#再 /etc/profile 内添加
export NAMESRV_ADDR=172.17.10.50:9876

#生效
source  /etc/profile

2、普通订阅发布测试


常见一个主题:TopicTest (可在控制台创建)


启动生产者:

/opt/rocketmq/bin/tools.sh  org.apache.rocketmq.example.quickstart.Producer


启动消费者:

/opt/rocketmq/bin/tools.sh  org.apache.rocketmq.example.quickstart.Consumer


3、压力测试

压力测试一定要进入命令目录下才可以


在使用压力测试命令之前,需要给权限

chmod +777  -R  /opt/rocketmq/benchmark/
cd /opt/rocketmq/benchmark/


启动消费端: 启动时需要指定主题; 创建主题

TopicTest

./consumer.sh  -t  TopicTest  -n 172.17.10.50:9876;172.17.10.51:9876
# -n 指定 nameserver 地址 -t 指定主题
# -g 可指定组

#异常
./runclass.sh:行71: /usr/java/bin/java: 没有那个文件或目录

#更改JAVA_HOME为java目录
65 if [ -z "$JAVA_HOME" ]; then
66   JAVA_HOME=/usr/
67 fi
68
69 JAVA="$JAVA_HOME/bin/java"


启动生产端:

./producer.sh     -t  TopicTest  -n 172.17.10.50:9876;172.17.10.51:9876

停止压测:

./shutdown.sh  
#可能会不好使,直接退出终端

#本地模拟的环境,非真实生产环境

异常:system busy 与 broker busy

参考:

https://www.cnblogs.com/enenen/p/10138511.html

broker busy异常: 可通过增大 waitTimeMillsInSendQueue 解决

system busy异常:可通过增大 osPageCacheBusyTimeOutMills 解决

#发送队列等待时间
waitTimeMillsInSendQueue=3000
#系统页面缓存繁忙超时时间(翻译),默认值 1000
osPageCacheBusyTimeOutMills=5000



版权声明:本文为m0_49962289原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。