Rsync实现负载均衡的数据同步

  • Post author:
  • Post category:其他


使用三台服务器:

系统:CentOS 6.8

192.168.8.169 开发服务器

192.168.8.167 线上服务器1

192.168.8.168 线上服务器2

实现思路:

在开发服务器上制定一个规则,

即只要rsync.txt存在,

线上服务器就开始进行文件同步,同步完删除该文件。

实现步骤:

(1)安装Rsync。

1、Rsync简介:

Rsync(remote synchronize)是一个远程数据同步工具,可通过LAN/WAN快速同步多台主机间的文件。

Rsync使用所谓的“Rsync算法”来使本地和远 程两个主机之间的文件达到同步,

这个算法只传送两个文件的不同部分,而不是每次都整份传送,因此速度相当快。

2、Rsync安装:

wget https://download.samba.org/pub/rsync/rsync-3.1.3.tar.gz

tar -zxvf rsync-3.1.3.tar.gz

cd rsync-3.1.3

./configure –prefix=/usr/local/rsync

make

make install

第二步:

安装inotify和inotify-tools


Centos 6和CentOS 7,已经默认安装了inotify,

如果要查看是否安装,可以使用如下命令

:

ll /proc/sys/fs/inotify


如果列出如下三项,则证明已经安装

max_queued_events

max_user_instances

max_user_watches

第三步:

安装inotify-tools工具。

wget https://jaist.dl.sourceforge.net/project/inotify-tools/inotify-tools/3.13/inotify-tools-3.13.tar.gz

tar -zxvf inotify-tools-3.13.tar.gz

cd inotify-tools-3.13

./configure

make && make install


第四步:

线上服务器的配置:

touch /etc/rsyncd.conf

vim /etc/rsyncd.conf

输入以下内容:

uid = nobody

gid = nobody

use chroot = no

max connections = 10

strict mode = no

pid file = /var/run/rsyncd.pid

lock file = /var/run/rsync.lock

log file = /usr/data/rsync/rsyncd.log


[source-code]

path = /usr/local/nginx/html/hello/

comment = this is a comment message

ignore errors

read only = no

write noly = no

hosts allow = 192.168.8.169

hosts deny = *

list = false

uid = root

gid = root

auth users = root

secrets file = /usr/local/rsync/conf/server.pass

[source-code-update]

path = /usr/local/nginx/html/hello-update/

comment = this is a comment message

ignore errors

read only = no

write noly = no

hosts allow = 192.168.8.169

hosts deny = *

list = false

uid = root

gid = root

auth users = root

secrets file = /usr/local/rsync/conf/server.pass

说明:

source-code是源代码目录。

source-code-update 是控制是否更新的目录。

然后继续在线上服务器操作:

因为nginx部署的web位置一样,

cd /usr/local/nginx/html/

mkdir -p hello/

cd /usr/local/rsync(rsync的安装位置)

mkdir -p conf/

cd conf

touch server.pass

vim server.pass

root:123abc+-

给密码文件设置访问权限:

chmod 600 server.pass


第五步:

然后对开发服务器进行配置:

touch /etc/rsyncd.conf

vim /etc/rsyncd.conf

输入以下内容:

uid = nobody

gid = nobody

use chroot = no

max connections = 10

strict mode = no

pid file = /var/run/rsyncd.pid

lock file = /var/run/rsync.lock

log file = /usr/data/rsync/rsyncd.log

[source-code]

path = /usr/local/nginx/html/hello/

comment = this is a comment message

ignore errors

read only = no

write noly = no

hosts allow = 192.168.8.169,192.168.8.167

hosts deny = *

list = false

uid = root

gid = root

auth users = root

secrets file = /usr/local/rsync/conf/server.pass

建立项目目录

cd /usr/local/nginx/html/

mkdir hello

cd /usr/local/rsync(rsync的安装位置)

mkdir -p conf/

然后还要建立一个密码文件’/usr/local/rsync/conf/server.pass’ 由于1是客户端,因此密码没有前缀

123abc+-

给密码文件设置访问权限:

chmod 600 server.pass


cp /usr/local/rsync/bin/rsync /usr/bin/

/usr/local/rsync/bin/rsync –daemon

sh datarsync.sh


第一次测试

/usr/local/bin/inotifywait \

-mrq –timefmt ‘%d/%m/%y’ \

–format ‘%T %w%f%e’ \

-e modify,delete,create,attrib \

/usr/local/nginx/html/hello/index.html \

结果:有改变时,有输出

第二次测试,输出到log:

/usr/local/bin/inotifywait \

-mrq –timefmt ‘%d/%m/%y’ \

–format ‘%T %w%f%e’ \

-e modify,delete,create,attrib \

/usr/local/nginx/html/hello/ | while read files

do

echo “${files} was rsyncd” >>/tmp/rsync.log 2>&1

done

结果:日志中有信息

第三次测试:

#!/bin/bash

src=/usr/local/nginx/html/hello/

user=root

host2=192.168.8.167

$dst2=source-code

/usr/local/bin/inotifywait \

-mrq –timefmt ‘%d/%m/%y’ \

–format ‘%T %w%f%e’ \

-e modify,delete,create,attrib \

/usr/local/nginx/html/hello/ | while read files

do

/usr/bin/rsync -vzrtopg –delete –progress –password-file=/usr/local/rsync/conf/server.pass $src $user@$host2::$dst2

echo “${files} was rsyncd” >>/tmp/rsync.log 2>&1

done

/usr/bin/rsync -vzrtopg –progress –delete root@192.168.8.169::source-code/usr/local/nginx/html/hello/* /usr/local/nginx/html/hello/


使用/usr/local/bin/inotifywait命令报错:

/usr/local/bin/inotifywait: error while loading shared libraries: libinotifytool

s.so.0: cannot open shared object file: No such file or directory

[root@db zzh]# ll /proc/sys/fs/inotify (如果有下列三项则支持inotifytools)

total 0

-rw-r–r– 1 root root 0 Sep 20 16:52 max_queued_events

-rw-r–r– 1 root root 0 Sep 20 16:52 max_user_instances

-rw-r–r– 1 root root 0 Sep 20 16:52 max_user_watches

解决:

ln -s /usr/local/lib/libinotifytools.so.0 /usr/lib64/libinotifytools.so.0

rsync -avzP root@192.168.8.169::source-code /data/test/

rsync常见错误

https://blog.51cto.com/loveyan/713816


169开发环境

166负载均衡服务器

167 web服务器1

168 web服务器2

169 修改代码同步到167和168上。

169作为rsync客户端,

167和168作为rsync服务端,

服务端配置(即167和168的配置):

(1)服务安装

yum install rsync xinetd

(2)为 rsyncd 服务编辑配置文件,默认没有,需自己编辑

vim /etc/rsyncd.conf

写入以下内容:

uid = root

gid = root

use chroot = no

max connections = 5

timeout = 600

pid file = /var/run/rsyncd.pid

lockfile = /var/run/rsyncd.lock

log file = /var/log/rsyncd.log

[web1]

path = /usr/local/nginx/html/hello/

ignore errors = yes

read only = no

write only = no

hosts allow = 192.168.8.169

hosts deny = *

list = yes

auth users = web

secrets file = /etc/web.passwd

(3)创建文件同步的目录,上面配置里的path,如果有就不用创建了

mkdir /usr/local/nginx/html/hello/

(4)创建配置中的密码文件,并增加权限:

echo “web:123” > /etc/web.passwd

chmod 600 /etc/web.passwd

(5)重新启动

service xinetd restart


客户端配置(即169):

(1)安装软件

yum -y install rsync

(2)创建web目录

mkdir /usr/local/nginx/html/hello/

(3)设置密码并设置权限

echo “123”> /tmp/rsync.password

chmod 600 /tmp/rsync.password

测试:

rsync -avzP –delete –password-file=/tmp/rsync.password /usr/local/nginx/html/hello/ web@192.168.8.167::web1


数据实时同步

环境:Rsync + Inotify-tools。

wget https://jaist.dl.sourceforge.net/project/inotify-tools/inotify-tools/3.13/inotify-tools-3.13.tar.gz

tar -zxvf inotify-tools-3.13.tar.gz

mkdir /usr/local/inotify

cd inotify-tools-3.13

./configure –prefix=/usr/local/inotify/

make && make install

设置环境变量

# vim /root/.bash_profile

export PATH=/usr/local/inotify/bin/:$PATH

# source /root/.bash_profile

# echo ‘/usr/local/inotify/lib’ >> /etc/ld.so.conf –加载库文件

# ldconfig

# ln -s /usr/local/inotify/include /usr/include/inotify

vim /etc/profile

在末尾增加一行:

export PATH=$PATH:/usr/local/inotify/bin

使配置生效:

source /etc/profile

创建shell文件:

vim /test.sh

输入以下内容:

#!/bin/bash

src=/usr/local/nginx/html/hello/

user=web

host1=192.168.8.167

dst1=web1

passpath=/tmp/rsync.password

/usr/local/inotify/bin/inotifywait \

-mrq –timefmt ‘%d/%m/%y’ \

–format ‘%T %w%f%e’ \

-e modify,delete,create,attrib \

/usr/local/nginx/html/hello/ | while read files

do

rsync -vzrtopg –delete –progress –passfile=$passfile-path $src $user@$host1::$dst1

echo “${files} was rsyncd” >>/tmp/rsync.log 2>&1

done

chmod 755 /data/test/test.sh

# /data/test/test.sh &

# echo ‘/data/test/test.sh &’ >> /etc/rc.local –设置开机自启

转载于:https://www.cnblogs.com/gyfluck/p/11512065.html