为了账号安全,请及时绑定邮箱和手机立即绑定

LVS+keepalived+DRBD+heartbeat+mysql

标签:
MySQL


wKiom1XQKIaC4YcsAAH6io_UKl8752.jpg

IP地址规划:

LVS:

10.1.1.23 主LVS+keepalived

10.1.1.22 备LVS+keepalived

主mysql:

10.1.1.5  DRBD+heartbeat

10.1.1.6  DRBD+heartbeat

10.1.1.7  VIP(heartbeat)

10.1.1.10 VIP(lvs)

从mysql+nginx:

10.1.1.102 Ng+php

10.1.1.102 Ng+php

VIP1:10.1.1.8 (nginx)

VIP2: 10.1.1.9 (mysql)

安装DRBD+mysql+heartbeat

安装DRBD

1.1.规划

10.1.1.5(主)  /data

10.1.1.6(备)  /data

2.安装(主备安装一致):

2.1 安装依赖:

yum install kernel-devel kernel-headers

2.2.编译安装

tar -zxf drbd-8.3.16.tar.gz 

./configure --prefix=/usr/local/drbd --with-km

make && make install

2.3 拷贝配置文件模板

mv /usr/local/drbd/etc/drbd.conf /usr/local/drbd/drbd.conf.old

cp /tmp/soft/drbd-8.3.16/scripts/drbd.conf.exampl /usr/local/drbd/etc/drbd.conf

2.4 设置链接

ln -s /usr/local/drbd/etc/drbd.conf /etc/drbd.conf

ln -s /usr/local/drbd/etc/rc.d/init.d/drbd /etc/init.d/drbd

3.配置文件(主备一致):

global {

    usage-count no;

}

common {

  syncer { rate 200M; }

}

resource r0 {

  protocol C;

  handlers {

    pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";

    pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";

    local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f";

    fence-peer "/usr/lib/heartbeat/drbd-peer-outdater -t 5";

    pri-lost "/usr/lib/drbd/notify-pri-lost.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";

    split-brain "/usr/lib/drbd/notify-split-brain.sh root";

    out-of-sync "/usr/lib/drbd/notify-out-of-sync.sh root";

  }

  startup {

    degr-wfc-timeout 120;

    outdated-wfc-timeout 2;

  }

  disk {

    on-io-error   detach;

    fencing resource-only;

  }

  net {

    cram-hmac-alg "sha1";

    shared-secret "ckl893";

  }

  on ckl-bd1 {

    device     /dev/drbd0;

    disk       /dev/sda5;

    address    10.1.1.5:7788;

    flexible-meta-disk  internal;

  }

  on ckl-bd2 {

    device    /dev/drbd0;

    disk      /dev/sda5;

    address   10.1.1.6:7788;

    meta-disk internal;

  }

}

4.创建数据块:

(主备一致)

#dd if=/dev/zero bs=1M count=1 of=/dev/sda5;sync

#drbdadm create-md r0

5.启动DRBD服务

#umount /data

#/etc/init.d/drdb start

6.配置主服务,格式化DRBD(10.1.1.5):

#drbdsetup /dev/drbd1 primary –o

# mkfs.ext4 /dev/drbd0

7.主挂载DRBD,创建文件(10.1.1.5):

#mount /dev/drbd0 /mnt

#cd /mnt

#touch nihao.txt

8.测试同步:

停掉主(10.1.1.5):

#/etc/init.d/drbd stop

从设置为主(10.1.1.6):

#drbdsetup /dev/drbd1 primary –o

# mount /dev/drbd0 /mnt

#cd /mnt

# ll nihao.txt 

-rw-r--r-- 1 root root 0 May  7 10:42 nihao.txt

9.切换回来:

10.1.1.6上执行:

#umount /mnt

#drbdadm secondary all

10.1.1.5 上执行:

#drbdadm primary all

# mount /dev/drbd0 /mnt

安装mysql

1.mysql安装规划:

10.1.1.5(主):/mnt DRBD挂载的分区

10.1.1.6(备)

2.首先启动主DRBD(10.1.1.5)挂载DRBD到/mnt

安装mysql到/mnt 下

3.备(10.1.1.6)切换到主,挂载DRBD 到/mnt

安装mysql到/mnt 下

4.切换回

10.1.1.5主

10.1.1.6备

安装heartbeat

规划:10.1.1.5 (主)

10.1.1.6 (备)

10.1.1.7 (heartbeat VIP)

      主备安装一致:

1.安装heartbeat 下载需要的源:

wget ftp://mirror.switch.ch/pool/1/mirror/scientificlinux/6rolling/x86_64/os/Packages/epel-release-6-5.noarch.rpm

2.安装heartbeat:

yum -y install heartbeat*

3.配置heartbeat

3.1.拷贝模板配置文件:

cp /usr/share/doc/heartbeat-3.0.4/ha.cf /etc/ha.d/

cp /usr/share/doc/heartbeat-3.0.4/authkeys /etc/ha.d/

cp /usr/share/doc/heartbeat-3.0.4/haresources /etc/ha.d/

4.heartbeat主配置文件(主备一致):

cat /etc/ha.d/ha.cf

logfile /var/log/ha-log

logfacility     local0

keepalive 2

deadtime 30

warntime 10

initdead 120

udpport 694

baud    19200

bcast   eth0

auto_failback on

watchdog /dev/watchdog

node    ckl-bd1

node    ckl-bd2

ping 10.1.1.1

respawn hacluster /usr/lib64/heartbeat/ipfail

apiauth ipfail gid=haclient uid=hacluster

5.heartbeat认证文件(主备一致):

cat /etc/ha.d/authkeys

auth 1

1 sha1 ckl893!

6.资源文件

10.1.1.5(主):

# cat /etc/ha.d/haresources

ckl-bd1 drbddisk::r0 Filesystem::/dev/drbd0::/mnt mysql IPaddr::10.1.1.7/24/eth0:1

10.1.1.6(备):

# cat /etc/ha.d/haresources

ckl-bd2 drbddisk::r0 Filesystem::/dev/drbd0::/mnt mysql IPaddr::10.1.1.7/24/eth0:1

drbdisk脚本需要添加到/etc/ha.d/resource.d 下

7.测试heartbeat+mysql+DRBD

主,停掉mysql,停掉heartbeat

从监测到后,挂载DRBD,启动mysql

配置mysql主从同步

10.1.1.107 主

10.1.1.102 从

10.1.1.103 从

配置简单略过

安装LVS

1.安装lvs

10.1.1.23 

10.1.1.22

执行同样操作:

1.1.安装需要包:

yum -y install popt popt-devel popt-static libnl libnl-devel

1.2.lvs在2.4之后内核自带,查看是否已经存在:

# cat /boot/config-2.6.32-573.el6.x86_64 | grep -i -A 15 "ipvs"

# IPVS transport protocol load balancing support

#

CONFIG_IP_VS_PROTO_TCP=y

CONFIG_IP_VS_PROTO_UDP=y

CONFIG_IP_VS_PROTO_AH_ESP=y

CONFIG_IP_VS_PROTO_ESP=y

CONFIG_IP_VS_PROTO_AH=y

CONFIG_IP_VS_PROTO_SCTP=y

#

# IPVS scheduler

#

CONFIG_IP_VS_RR=m

CONFIG_IP_VS_WRR=m

CONFIG_IP_VS_LC=m

CONFIG_IP_VS_WLC=m

CONFIG_IP_VS_LBLC=m

CONFIG_IP_VS_LBLCR=m

CONFIG_IP_VS_DH=m

CONFIG_IP_VS_SH=m

CONFIG_IP_VS_SED=m

CONFIG_IP_VS_NQ=m

#

# IPVS application helper

#

CONFIG_IP_VS_FTP=m

CONFIG_IP_VS_PE_SIP=m

#

# IP: Netfilter Configuration

#

CONFIG_NF_DEFRAG_IPV4=m

CONFIG_NF_CONNTRACK_IPV4=m

# CONFIG_NF_CONNTRACK_PROC_COMPAT is not set

CONFIG_IP_NF_QUEUE=m

CONFIG_IP_NF_IPTABLES=m

CONFIG_IP_NF_MATCH_ADDRTYPE=m

CONFIG_IP_NF_MATCH_AH=m

CONFIG_IP_NF_MATCH_ECN=m

1.3.说明ipvs:

IPVS transport protocol:ipvs支持的协议,tcp、udp、ah、esp、sctp

ipvs scheduler:ipvs调度模式,RR、WRR、LC、WLC、LBLC、LBLCR、DH、SH、SED、NQ

1.4.安装lvsadmin(lvs管理工具):

tar -zxvf ipvsadm-1.26.tar.gz

cd ipvsadm-1.26

make && make install

2.安装keepalived

2.1.安装需要包:

yum -y install openssl-devel popt-devel libnl* libpopt* popt-static

ln -s /usr/src/kernels/2.6.32-279.el6.x86_64/ /usr/src/linux

2.2.安装lvs

tar -zxvf keepalived-1.2.4.tar.gz 

cd keepalived-1.2.4

./configure --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --mandir=/usr/share/man --infodir=/usr/share/info --sharedstatedir=/usr/com --with-kernel-dir=/usr/src/linux

make && make install

3.配置keepalived

3.1.在10.1.1.23 上配置:

# vi /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {

   notification_email {

        kangly@cosyjoy.com

   }

   notification_email_from Alexandre.Cassen@firewall.loc

   smtp_server 10.1.1.23

   smtp_connect_timeout 30

   router_id LVS_MASTER

}

vrrp_instance VI_1 {

    state MASTER

    interface eth0

    virtual_router_id 77 

    priority 100

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass ckl893

    }

    virtual_ipaddress {

       10.1.1.8/24   #nginx VIP

       10.1.1.9/24   #mysql slave VIP

       10.1.1.10/24  #mysql 主VIP

    }

}

virtual_server 10.1.1.8 80 {

    delay_loop 6

    lb_algo rr 

    lb_kind DR 

    persistence_timeout 50

    protocol TCP

    real_server 10.1.1.102 80 {

        weight 1

        TCP_CHECK {

            connect_timeout 10

            nb_get_retry 3

            delay_before_retry 3

        }

    }

    real_server 10.1.1.103 80 {

        weight 1

        TCP_CHECK {

            connect_timeout 3

            nb_get_retry 3

            delay_before_retry 3

        }

    }

}

virtual_server 10.1.1.9 3306 {

    delay_loop 30

    lb_algo rr

    lb_kind DR

    persistence_timeout 120

    protocol TCP

    real_server 10.1.1.102 3306 {

        weight 1

                MISC_CHECK {

                misc_path "/etc/keepalived/check_slave.pl 10.1.1.102"

                misc_dynamic

        }

    }

    real_server 10.1.1.103 3306 {

        weight 1

                MISC_CHECK {

                misc_path "/etc/keepalived/check_slave.pl 10.1.1.103"

                misc_dynamic

        }

    }

}

3.2.配置备lvs

在10.1.1.22 上:

主从配置文件不一致

    state BACKUP  状态为备份

    priority 90  优先级小于主

4.本地添加VIP脚本:

10.1.1.5

10.1.1.6 

# cat /etc/init.d/lvsrs 

#!/bin/bash

#description : start realserver

VIP=10.1.1.10

. /etc/rc.d/init.d/functions

case "$1" in

    start)

        echo " start LVS  of  REALServer"

 /sbin/ifconfig lo:0 $VIP broadcast $VIP netmask 255.255.255.255 up

        echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore

        echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce

        echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore

        echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

        ;;

    stop)

        /sbin/ifconfig lo:0 down

        echo "close LVS REALserver"

        echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore

        echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce

        echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore

        echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce

        ;;

    *)

        echo "Usage: $0 {start|stop}"

        exit 1

esac

5.本地添加ip脚本:

10.1.1.102

10.1.1.103

#!/bin/bash

#description : start realserver

VIP1=10.1.1.8

VIP2=10.1.1.9

. /etc/rc.d/init.d/functions

case "$1" in

    start)

        echo " start LVS  of  REALServer"

 /sbin/ifconfig lo:0 $VIP1 broadcast $VIP1 netmask 255.255.255.255 up

 /sbin/ifconfig lo:1 $VIP2 broadcast $VIP2 netmask 255.255.255.255 up

        echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore

        echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce

        echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore

        echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

        ;;

    stop)

        /sbin/ifconfig lo:0 down

        echo "close LVS REALserver"

        echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore

        echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce

        echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore

        echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce

        ;;

    *)

        echo "Usage: $0 {start|stop}"

        exit 1

esac

arp相关参数说明:

arp_annonuce(是否将本机地址网络通过arp通告给其它网络):

    0:通告本机所有地址接口的网络。

    1:尽量只通告本接口地址的网络。

    2:只通告连接本地借口地址网络。

arg_ignore(是否响应其它的arp请求):

    0:在接受到请求后,将本地所有地址响应出去。

    1:请求报文从哪个接口进入,就必须使用此接口配置的地址才能响应。

6.mysql 同步检测脚本:

注意事项:执行用户必须有show slave status 权限

# cat check_slave.pl 

#!/usr/bin/perl -w

use DBI;

use DBD::mysql;

$SBM = 120;

$db = "cookbook";

$host = $ARGV[0];

$port = 3306;

$user = "root";

$pw = 'yiqu@123'; #这里是单引号,因为有@

$query = "show slave status";

$dbh = DBI->connect("DBI:mysql:$db:$host:$port", $user, $pw, { RaiseError => 0,PrintError => 0 });

if (!defined($dbh)) {

    exit 1;

}

$sqlQuery = $dbh->prepare($query);

$sqlQuery->execute;

$Slave_IO_Running =  "";

$Slave_SQL_Running = "";

$Seconds_Behind_Master = "";

while (my $ref = $sqlQuery->fetchrow_hashref()) {

    $Slave_IO_Running = $ref->{'Slave_IO_Running'};

    $Slave_SQL_Running = $ref->{'Slave_SQL_Running'};

    $Seconds_Behind_Master = $ref->{'Seconds_Behind_Master'};

}

$sqlQuery->finish;

$dbh->disconnect();

if ( $Slave_IO_Running eq "No" || $Slave_SQL_Running eq "No" ) {

    exit 1;

} else {

    if ( $Seconds_Behind_Master gt $SBM ) {

        exit 1;

    } else {

        exit 0;

    }

}

7.启动LVS

# /etc/init.d/keepalived start

查看LVS路由表信息:

#ipvsadm -Ln

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  10.1.1.8:80 rr persistent 50

  -> 10.1.1.102:80                Route   1      0          0         

  -> 10.1.1.103:80                Route   1      0          0         

TCP  10.1.1.9:3306 rr persistent 120

  -> 10.1.1.102:3306              Route   124    0          0         

  -> 10.1.1.103:3306              Route   124    0          0         

TCP  10.1.1.10:3306 rr persistent 120

  -> 10.1.1.7:3306                Route   1      0          0

8.测试web

停掉:10.1.1.102 nginx

# /etc/init.d/nginx stop

查看LVS路由表信息:

# ipvsadm -Ln

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  10.1.1.8:80 rr persistent 50

  -> 10.1.1.103:80                Route   1      0          0         

TCP  10.1.1.9:3306 rr persistent 120

  -> 10.1.1.102:3306              Route   124    0          0         

  -> 10.1.1.103:3306              Route   124    0          0         

TCP  10.1.1.10:3306 rr persistent 120

  -> 10.1.1.7:3306                Route   1      0          0

路由表删除了10.1.1.102 的信息;

启动10.1.1.102 nginx

 # /etc/init.d/nginx start

查看LVS路由表信息:

# ipvsadm -Ln

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  10.1.1.8:80 rr persistent 50

  -> 10.1.1.102:80                Route   1      0          0         

  -> 10.1.1.103:80                Route   1      0          0         

TCP  10.1.1.9:3306 rr persistent 120

  -> 10.1.1.102:3306              Route   124    0          0         

  -> 10.1.1.103:3306              Route   124    0          0         

TCP  10.1.1.10:3306 rr persistent 120

  -> 10.1.1.7:3306                Route   1      0          0

9.测试mysql读

停止10.1.1.102 mysql

# /etc/init.d/mysqld stop

查看路由表信息:

# ipvsadm -Ln        

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  10.1.1.8:80 rr persistent 50

  -> 10.1.1.102:80                Route   1      0          0         

  -> 10.1.1.103:80                Route   1      0          0         

TCP  10.1.1.9:3306 rr persistent 120

  -> 10.1.1.103:3306              Route   124    0          0         

TCP  10.1.1.10:3306 rr persistent 120

  -> 10.1.1.7:3306                Route   1      0          0

已经剔除了10.1.1.102 mysql

启动10.1.1.102 mysql

# /etc/init.d/mysqld start

查看路由表信息:

# ipvsadm -Ln

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  10.1.1.8:80 rr persistent 50

  -> 10.1.1.102:80                Route   1      0          0         

  -> 10.1.1.103:80                Route   1      0          0         

TCP  10.1.1.9:3306 rr persistent 120

  -> 10.1.1.103:3306              Route   124    0          0         

TCP  10.1.1.10:3306 rr persistent 120

  -> 10.1.1.7:3306                Route   1      0          0

测试检测同步脚本:

停掉10.1.1.102 同步:

>slave stop;

查看路由表信息:

# ipvsadm -Ln        

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  10.1.1.8:80 rr persistent 50

  -> 10.1.1.102:80                Route   1      0          0         

  -> 10.1.1.103:80                Route   1      0          0         

TCP  10.1.1.9:3306 rr persistent 120

  -> 10.1.1.103:3306              Route   124    0          0         

TCP  10.1.1.10:3306 rr persistent 120

  -> 10.1.1.7:3306                Route   1      0          0

启动同步

>slave start;

查看路由表信息:

# ipvsadm -Ln

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  10.1.1.8:80 rr persistent 50

  -> 10.1.1.102:80                Route   1      0          0         

  -> 10.1.1.103:80                Route   1      0          0         

TCP  10.1.1.9:3306 rr persistent 120

  -> 10.1.1.103:3306              Route   124    0          0         

TCP  10.1.1.10:3306 rr persistent 120

  -> 10.1.1.7:3306                Route   1      0          0

错误信息:

[root@localhost keepalived]# tail -f /var/log/messages

May  6 11:00:03 localhost Keepalived_vrrp[4233]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 10.1.1.8

May  6 11:00:10 localhost Keepalived_vrrp[4233]: receive an invalid ip number count associated with VRID!

May  6 11:00:10 localhost Keepalived_vrrp[4233]: bogus VRRP packet received on eth0 !!!

May  6 11:00:10 localhost Keepalived_vrrp[4233]: VRRP_Instance(VI_1) Dropping received VRRP packet...

May  6 11:00:11 localhost Keepalived_vrrp[4233]: receive an invalid ip number count associated with VRID!

解决办法:修改virtual_router_id 51 为另一个值,如:virtual_router_id 77

change master to master_host='10.1.1.7',master_port=3306,master_user='repl',master_password='slavepass',master_log_file='mysql-bin.000017',master_log_pos=881;

perl Can't locate DBI.pm

yum -y install perl perl-DBI  perl-DBD-MySQL

©著作权归作者所有:来自51CTO博客作者ckl893的原创作品,如需转载,请注明出处,否则将追究法律责任


点击查看更多内容
TA 点赞

若觉得本文不错,就分享一下吧!

评论

作者其他优质文章

正在加载中
  • 推荐
  • 评论
  • 收藏
  • 共同学习,写下你的评论
感谢您的支持,我会继续努力的~
扫码打赏,你说多少就多少
赞赏金额会直接到老师账户
支付方式
打开微信扫一扫,即可进行扫码打赏哦
今天注册有机会得

100积分直接送

付费专栏免费学

大额优惠券免费领

立即参与 放弃机会
意见反馈 帮助中心 APP下载
官方微信

举报

0/150
提交
取消