Thursday, May 14, 2015

Setup HAproxy and keepalived to ip failover to an EAP cluster in RHEL7



Haproxy is the hardware load balancer that most commonly used by the industry. Also it is a later 7 load balancer which means you can load balance many (any) backend via haproxy load balancer.
But there will be a problem when you many backend using single load balancer node. Because there is a possibility of fail the load balancer. Then whole system will stop and you have wait until the server up and run or you have to setup new load balancer immediately. Because of that you have to have a mechanism to overcome this problem.

We can setup keeplived to face ip failover which is ship separately with haproxy. To setup keepalived we have to have two haproxy load balancer servers. And we need virtual ip (VIP) to front load balancers.

In this post I’m going to show how to load balance two eap servers via failover haproxy load balancers through keepalived.


Lb1,
Eth0 – xxx.xxx.xxx.12
Eth1 – xxx.xxx.xxx.17 (VIP)
Lb2,
Eth0 – xxx.xxx.xxx.13
Eth1 – xxx.xxx.xxx.17 (VIP)
Eap1 & eap2,
xxx.xxx.xxx.14

Before starting configurations on haproxy and keepalived there are some common things to consider.
First one is selinux in linux machine. Since haproxy is hardware load balancer we have to insert selinux rules before start working. Check this post I previously posted how to add selinux policies to enable ports.

But in this time I’m going to disable selinux in my haproxy machines. Do below exercise to disable selinux. 

[root@localhost  ~]# vi /etc/sysconfig/selinux

Now reboot the machine to apply changes

[root@localhost  ~]# reboot

Second consideration is firewall setting of the machine. I am going to disable firewall to make easy configurations,


Now install haproxy and keepalived in two load balancer machines,

 [root@localhost ~]# yum install keepalived
[root@localhost  ~]# yum install haproxy

Now install two EAPs in different servers.
Install jboss eap using jar installer (read how to installjboss eap using installer). Click to view how to install jboss eap
Now you have to setup vip in load balancer servers.
Setting up vip in haproxy cluster
To do this you have to have separate two Ethernet connections in both load balancer servers.
First one is for the machine ip and second one is for virtual ip.

[root@localhost  ~]# nmtui


After done configurations in virtual ip settings, now setup haproxy configuration files in both load balancers,
Edit and insert below configurations to the haproxy conf file,
[root@localhost  ~]# vi /etc/haproxy/haproxy.cfg
And insert below code,
#---------------------------------------------------------------------
# Example configuration for a possible web application.  See the
# full configuration options online.
#
#   http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
#
#---------------------------------------------------------------------

#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
    # to have these messages end up in /var/log/haproxy.log you will
    # need to:
    #
    # 1) configure syslog to accept network log events.  This is done
    #    by adding the '-r' option to the SYSLOGD_OPTIONS in
    #    /etc/sysconfig/syslog
    #
    # 2) configure local2 events to go to the /var/log/haproxy.log
    #   file. A line like the following can be added to
    #   /etc/sysconfig/syslog
    #
    #    local2.*                       /var/log/haproxy.log
    #
    log         127.0.0.1 local0

    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon

    # turn on stats unix socket
    stats socket /var/lib/haproxy/stats

#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
    mode                    http
    log                     global
    option                  httplog
    option                  dontlognull
    option http-server-close
    option forwardfor       except 127.0.0.0/8
    option                  redispatch
    retries                 3
    timeout http-request    10s
    timeout queue           1m
    timeout connect         10s
    timeout client          1m
    timeout server          1m
    timeout http-keep-alive 10s
    timeout check           10s
    maxconn                 3000

#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
listen webfarm *:80
       mode http
       stats enable
       stats uri /haproxy?stats
       stats realm Haproxy\ Statistics
       stats auth haproxy:stats
#      balance roundrobin
       balance source
       cookie LBN insert indirect nocache
       option httpclose
       option forwardfor
       server eap1 xxx.xxx.xxx.14:8080 cookie node1 check
       server eap2 xxx.xxx.xxx.14:8081 cookie node1 check

Start haproxy after done configurations,

[root@localhost  ~]# systemctl start haproxy.service

Also you can log in to the haproxy management panel using below url in browser


Then it will prompt to add username and password,


Give haproxy as username and stats as the password and click ok.


Now change keepalived configuration file in both servers,
In lb1,

#
# /etc/keepalived/keepalived.conf
#

global_defs {

  notification_email {
    mymails@mycompany.com
    root@localhost
  }

  notification_email_from lb1@haproxycluster
  router_id HAPROXY
}

vrrp_script chk_haproxy {
  script   "killall -0 haproxy"
  interval 2
  weight   2
}

vrrp_instance LB_VIP {
  state MASTER
  interface eno16780032 #this can be eth0
  virtual_router_id 42
  # Higher priority on other node
  priority          101
  advert_int        1
  # notify "/usr/local/bin/VRRP-notification.sh"

    virtual_ipaddress {
        xxx.xxx.xxx.17/24 dev eno33559296 #this can be eth1
    }

  track_script {
    chk_haproxy
  }

  authentication {
    auth_type PASS
    auth_pass 1111
}
}

In lb2,

#
# /etc/keepalived/keepalived.conf
#

global_defs {

notification_email {
    mymails@mycompany.com
    root@localhost
  }

  notification_email_from lb2@haproxycluster
  router_id HAPROXY
}

vrrp_script chk_haproxy {
  script   "killall -0 haproxy"
  interval 2
  weight   2
}

vrrp_instance LB_VIP {
# state BACKUP
  state MASTER
  interface eno16780032 #this can be eth0
  virtual_router_id 42
  # Higher priority on other node
  priority          100
  advert_int        1
  # notify "/usr/local/bin/VRRP-notification.sh"

    virtual_ipaddress {
        xxx.xxx.xxx.17/24 dev ens224 #this can be eth1
    }

  track_script {
    chk_haproxy
  }
  authentication {
    auth_type PASS
    auth_pass 1111
  }
}

****Note that if you are using virtual machines, remember to add Ethernet DEVICE NO instead of using Ethernet name.



Now you have successfully configured the haproxy load balancer with keeplived ip failover mechanism.

To test the setup, simply invoke below url using any accessible server and you can see EAP servers are working fine, 172.16.109.17 is the vip


You can achieve the same using DNS (domain name server) configurations other than using virtual IP (VIP). To do that you have add a single DNS entry for both two load balancers.