Categories
Computing

Intégration d’une ressource DRBD sur un cluster Pacemaker/CoroSync sous GNU/Linux Debian

Partitionnement

Remarque : les actions suivantes sont à réaliser sur les deux noeuds.

Voici le disque dédié à DRBD, sans partition :

# fdisk -l /dev/cciss/c0d1
Disk /dev/cciss/c0d1: 39.4 GB, 39406141440 bytes
255 heads, 32 sectors/track, 9432 cylinders
Units = cylinders of 8160 * 512 = 4177920 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x64476aa0

Disk /dev/cciss/c0d1 doesn't contain a valid partition table

Lancer l’outil de partitionnement de ce même disque :

# cfdisk /dev/cciss/c0d1

Une fois le partitionnement terminé, voici ce que nous obtenons :

# fdisk -l /dev/cciss/c0d1
Disk /dev/cciss/c0d1: 39.4 GB, 39406141440 bytes
255 heads, 32 sectors/track, 9432 cylinders
Units = cylinders of 8160 * 512 = 4177920 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x64476aa0


Device Boot      Start         End      Blocks   Id  System
/dev/cciss/c0d1p1               1        9432    38482544   83  Linux

DRBD ou le RAID 1 over TCP/IP

Remarque : les actions suivantes sont à réaliser sur les deux noeuds. Installer DRBD :

# aptitude install drbd8-utils

Lancer le module approprié :

# modprobe drbd

Vérifier la présence du module :

# lsmod | grep drbd
drbd                  172668  0
lru_cache               4018  1 drbd
cn                      3631  1 drbd

Sauvegarder la configuration d’origine :

# cp /etc/drbd.conf /etc/drbd.conf.original

Configurer DRBD :

# > /etc/drbd.conf
# vim /etc/drbd.conf
global { usage-count no; }
common { syncer { rate 100M; } }
resource r0 {
        protocol C;
        startup {
                wfc-timeout  15;
                degr-wfc-timeout 60;
        }
        net {
                cram-hmac-alg sha1;
                shared-secret "***********";
        }
        on vm-debian1 {
                device /dev/drbd0;
                disk /dev/cciss/c0d1;
                address 10.20.30.1:7788;
                meta-disk internal;
        }
        on vm-debian2 {
                device /dev/drbd0;
                disk /dev/cciss/c0d1;
                address 10.20.30.2:7788;
                meta-disk internal;
        }
}

Initialiser le RAID avec la ressource r0 :

# drbdadm create-md r0
md_offset 39406137344
al_offset 39406104576
bm_offset 39404900352

Found some data

 ==> This might destroy existing data! <==

Do you want to proceed?
[need to type 'yes' to confirm] yes

Writing meta data...
initializing activity log
NOT initialized bitmap
New drbd meta data block successfully created.

Démarrer le service DRBD :

# /etc/init.d/drbd start

Remarque  : les actions suivantes sont à réaliser uniquement sur le premier noeud.

Forcer la réplication des données de la ressource r0 depuis le premier noeud :

# drbdadm -- --overwrite-data-of-peer primary all

Vérifier le status de la ressource r0 :

# cat /proc/drbd

Créer le système de fichiers sur le volume DRBD :

# mkfs.ext3 /dev/drbd0

Intégration de la ressource DRBD sous Pacemaker

Créer le point de montage :

# mkdir /cluster

Exemple de configuration de Pacemaker :

# crm configure show
node vm-debian1
node vm-debian2
primitive ClusterDisk ocf:linbit:drbd \
        params drbd_resource="r0" \
        op monitor interval="15s"
primitive ClusterFS ocf:heartbeat:Filesystem \
        params device="/dev/drbd0" directory="/cluster" fstype="ext3"
primitive ClusterIP ocf:heartbeat:IPaddr2 \
        params ip="10.20.30.10" cidr_netmask="23" \
        op monitor interval="15"
primitive Nagios lsb:nagios \
        op monitor interval="15s" timeout="20s" \
        meta target-role="Stopped"
ms MasterSlaveDisk ClusterDisk \
        meta master-max="1" master-node-max="1" clone-max="2" clone-node-max="1" notify="true" target-role="Started"
colocation Nagios-with-ClusterFS inf: Nagios ClusterFS
colocation mount-ClusterFS-on-Master inf: ClusterFS MasterSlaveDisk:Master
colocation mount-ClusterIP-with-ClusterFS inf: ClusterIP ClusterFS
order Nagios-after-ClusterFS inf: ClusterFS Nagios
order mount-ClusterFS-after-ClusterDisk inf: MasterSlaveDisk:promote ClusterFS:start
order mount-ClusterIP-after-ClusterFS inf: ClusterFS ClusterIP
property $id="cib-bootstrap-options" \
        dc-version="1.0.9-74392a28b7f31d7ddc86689598bd23114f58978b" \
        cluster-infrastructure="openais" \
        expected-quorum-votes="2" \
        stonith-enabled="false" \
        no-quorum-policy="ignore" \
        last-lrm-refresh="1334323679"
rsc_defaults $id="rsc-options" \
        resource-stickiness="100"

Exemple d’un problème de synchronisation

Sur le noeud 1 :

# drbdadm status
<drbd-status version="8.3.7" api="88">
<resources config_file="/etc/drbd.conf">
<resource minor="0" name="r0" cs="StandAlone" ro1="Primary" ro2="Unknown" ds1="UpToDate" ds2="DUnknown" />
</resources>
</drbd-status>

Sur le noeud 2 :

# drbdadm status
<drbd-status version="8.3.7" api="88">
<resources config_file="/etc/drbd.conf">
<resource minor="0" name="r0" cs="StandAlone" ro1="Secondary" ro2="Unknown" ds1="Outdated" ds2="DUnknown" />
</resources>
</drbd-status>

Ce à quoi cela correspond :

You suffer from schizophrenic DRBD, better known as “split brain”.

Basically it means that both nodes became primary on your test resource and started writing to it. The result is an inconsistent resource, which is then called “split brain”. Please read the documentation carefully, you will have to discard the data on one of your nodes and let drbd sync that node from scratch.

Déconnecter le noeud avec les “mauvaises” données :

# drbdadm -- --discard-my-data connect all

Lancer la synchronisation depuis le noeud avec les “bonnes” données :

# drbdadm connect all
Categories
Computing

Installation et configuration d’un cluster Pacemaker/CoroSync sous GNU/Linux Debian 6 (Squeeze)

Installation et pré-configuration de CoroSync

remarque: les actions suivantes sont à réaliser sur les deux noeuds.

Installer les paquets nécessaires :

# aptitude install corosync pacemaker

Autoriser le démarrage de CoroSync :

# sed -i 's/START=no/START=yes/' /etc/default/corosync

Sauvegarder la configuration par défaut de CoroSync :

# cp /etc/corosync/corosync.conf /etc/corosync/corosync.conf.original

Modifier le fichier hosts :

# sed -i 's/^10.20.*$/10.20.30.1\tvm-debian1.france.local\tvm-debian1\n10.20.30.2\tvm-debian2.france.local\tvm-debian2/' /etc/hosts

Générer une paire de clés RSA pour SSH (sans mot de passe) :

# ssh-keygen -b 2048

Copier la clé publique sur le second noeud :

# if hostname | grep -q 1; then ssh-copy-id $(hostname | sed 's/1/2/'); else ssh-copy-id $(hostname | sed 's/2/1/'); fi

remarque: les actions suivantes sont à réaliser sur le premier noeud.

Générer des I/O :

# dd if=/dev/urandom of=/tmp/temp bs=1024 count=1000000&

En parallèle, générer une clé pour Corosync :

# corosync-keygen

Supprimer le fichier temporaire précédemment créé :

# rm /tmp/temp

Copier la clé précédemment générée sur le second noeud :

# scp /etc/corosync/authkey sfrafilx02:/etc/corosync/authkey

remarque: les actions suivantes sont à réaliser sur les deux noeuds.

Modifier le fichier de configuration de Corosync

# vim /etc/corosync/corosync.conf
# Please read the openais.conf.5 manual page

totem {
        version: 2

        # How long before declaring a token lost (ms)
        token: 3000

        # How many token retransmits before forming a new configuration
        token_retransmits_before_loss_const: 10

        # How long to wait for join messages in the membership protocol (ms)
        join: 60

        # How long to wait for consensus to be achieved before starting a new round of membership configuration (ms)
        consensus: 3600

        # Turn off the virtual synchrony filter
        vsftype: none

        # Number of messages that may be sent by one processor on receipt of the token
        max_messages: 20

        # Limit generated nodeids to 31-bits (positive signed integers)
        clear_node_high_bit: yes

        # Disable encryption
        secauth: off

        # How many threads to use for encryption/decryption
        threads: 0

        # Optionally assign a fixed node id (integer)
        # nodeid: 1234

        # This specifies the mode of redundant ring, which may be none, active, or passive.
        rrp_mode: none

        interface {
                # The following values need to be set based on your environment
                ringnumber: 0
                bindnetaddr: 10.20.30.1
                mcastaddr: 226.94.1.1
                mcastport: 5405
        }
}

amf {
        mode: disabled
}

service {
        # Load the Pacemaker Cluster Resource Manager
        ver:       0
        name:      pacemaker
        use_mgmtd: 1
}

aisexec {
        user:   root
        group:  root
}

logging {
        fileline: off
        to_stderr: yes
        to_logfile: yes
        logfile: /var/log/corosync.log
        logfile_priority: info
        to_syslog: no
        syslog_facility: daemon
        debug: off
        timestamp: on
        logger_subsys {
                subsys: AMF
                debug: off
                tags: enter|leave|trace1|trace2|trace3|trace4|trace6
        }
}

Démarrer CoroSync :

# /etc/init.d/corosync start

Vérifier l’état du cluster :

# crm_mon -1

Préparation des agents pour l’Open Cluster Framework (OCF)

Ajouter un agent OCF HTTPCheck :

# mkdir /usr/lib/ocf/resource.d/myscripts
# vim /usr/lib/ocf/resource.d/myscripts/HTTPCheck
#!/bin/sh
#
#
#       HTTPCheck OCF RA. It checks the accessibility of a Web resource.
#
# Copyright (c) 2004 SUSE LINUX AG, Lars Marowsky-Bré
#                    All Rights Reserved.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of version 2 of the GNU General Public License as
# published by the Free Software Foundation.
#
# This program is distributed in the hope that it would be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#
# Further, this software is distributed without any warranty that it is
# free of the rightful claim of any third person regarding infringement
# or the like.  Any license provided herein, whether implied or
# otherwise, applies only to this software file.  Patent licenses, if
# any, provided herein do not apply to combinations of this program with
# other software, or any other product whatsoever.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write the Free Software Foundation,
# Inc., 59 Temple Place - Suite 330, Boston MA 02111-1307, USA.
#
#######################################################################
# OCF parameters:
#   OCF_RESKEY_url    : Web resource URL
#
#######################################################################
# Initialization:

. ${OCF_ROOT}/resource.d/heartbeat/.ocf-shellfuncs

#######################################################################

meta_data() {
        cat <<END
<?xml version="1.0"?>
<!DOCTYPE resource-agent SYSTEM "ra-api-1.dtd">
<resource-agent name="HTTPCheck" version="0.9">
<version>1.0</version>

<longdesc lang="en">
This is HTTPCheck Resource Agent.
It checks the accessibility of a Web resource using wget binary.
Remember that you can configure wget options through /etc/wgetrc
configuration file.
</longdesc>
<shortdesc lang="en">HTTPCheck resource agent</shortdesc>

<parameters>

<parameter name="url" unique="0" required="1">
<longdesc lang="en">
This is a required parameter. This parameter specifies the Web resource to check.
</longdesc>
<shortdesc lang="en">Web resource</shortdesc>
<content type="string" default="www.debian.org" />
</parameter>

<parameter name="state" unique="1">
<longdesc lang="en">
Location to store the resource state in.
</longdesc>
<shortdesc lang="en">State file</shortdesc>
<content type="string" default="${HA_VARRUN}/OCF-{OCF_RESOURCE_INSTANCE}.state" />
</parameter>

</parameters>

<actions>
<action name="start"        timeout="15" />
<action name="stop"         timeout="15" />
<action name="monitor"      timeout="15" interval="10" depth="0" start-delay="0" />
<action name="meta-data"    timeout="5" />
<action name="validate-all"   timeout="30" />
</actions>
</resource-agent>
END
}

#######################################################################

httpcheck_usage() {
        cat <<END
usage: $0 {start|stop|monitor|validate-all|meta-data}

Expects to have a fully populated OCF RA-compliant environment set.
END
}

httpcheck_start() {
    httpcheck_monitor
    if [ $? =  $OCF_SUCCESS ]; then
        return $OCF_SUCCESS
    fi
    /usr/bin/wget -q -t 1 -T 4 --no-dns-cache --no-cache -4 -O ${OCF_RESKEY_state} $OCF_RESKEY_url
}

httpcheck_stop() {
    httpcheck_monitor
    if [ $? != $OCF_NOT_RUNNING ]; then
        rm ${OCF_RESKEY_state}
    fi
    return $OCF_SUCCESS
}

httpcheck_monitor() {
    # Monitor _MUST!_ differentiate correctly between running
    # (SUCCESS), failed (ERROR) or _cleanly_ stopped (NOT RUNNING).
    # That is THREE states, not just yes/no.

    if [ ! -f ${OCF_RESKEY_state} ]; then
        return $OCF_NOT_RUNNING
    else
        if /usr/bin/wget -q -t 1 -T 4 --no-dns-cache --no-cache -4 -O ${OCF_RESKEY_state} $OCF_RESKEY_url; then
            return $OCF_SUCCESS
        else
            return $OCF_ERR_GENERIC
        fi
    fi
}

httpcheck_validate() {

    # Is the state directory writable?
    state_dir=`dirname "$OCF_RESKEY_state"`
    touch "$state_dir/$$"
    if [ $? != 0 ]; then
        return $OCF_ERR_ARGS
    fi
    rm "$state_dir/$$"

    return $OCF_SUCCESS
}

: ${OCF_RESKEY_CRM_meta_interval=0}
: ${OCF_RESKEY_CRM_meta_globally_unique:="true"}

if [ "x$OCF_RESKEY_state" = "x" ]; then
    if [ ${OCF_RESKEY_CRM_meta_globally_unique} = "false" ]; then
        state="${HA_VARRUN}/OCF-${OCF_RESOURCE_INSTANCE}.state"

        # Strip off the trailing clone marker
        OCF_RESKEY_state=`echo $state | sed s/:[0-9][0-9]*\.state/.state/`
    else
        OCF_RESKEY_state="${HA_VARRUN}/OCF-${OCF_RESOURCE_INSTANCE}.state"
    fi
fi

if [ "x$OCF_RESKEY_url" = "x" ]; then
    OCF_RESKEY_url="www.debian.org"
fi

case $__OCF_ACTION in
meta-data)      meta_data;;
start)          httpcheck_start;;
stop)           httpcheck_stop;;
monitor)        httpcheck_monitor;;
validate-all)   httpcheck_validate;;
usage|help)     httpcheck_usage
                exit $OCF_SUCCESS
                ;;
*)              httpcheck_usage
                exit $OCF_ERR_UNIMPLEMENTED
                ;;
esac
rc=$?
ocf_log debug "${OCF_RESOURCE_INSTANCE} $__OCF_ACTION : $rc"
exit $rc
# chmod +x /usr/lib/ocf/resource.d/myscripts/HTTPCheck

Corriger l’agent OCF Squid :

# cp /usr/lib/ocf/resource.d/heartbeat/Squid /usr/lib/ocf/resource.d/heartbeat/Squid.original
# vim /usr/lib/ocf/resource.d/heartbeat/Squid
  ligne 198 : awk '/(tcp.*[0-9]+\.[0-9]+\.+[0-9]+\.[0-9]+:'$SQUID_PORT' |tcp.*:::'$SQUID_PORT' )/{

Configuration du cluster Pacemaker (CRM)

remarque: les actions suivantes sont à réaliser sur le premier noeud. Configurer les ressources du cluster :

# crm configure property stonith-enabled=false
# crm configure property no-quorum-policy=ignore
# crm configure rsc_defaults resource-stickiness=100
# crm configure primitive ClusterIP ocf:heartbeat:IPaddr2 \
      params ip=10.20.30.3 cidr_netmask=24 \
      op monitor interval=30
# crm configure primitive Proxy ocf:heartbeat:Squid \
      params squid_exe="/usr/sbin/squid3" squid_conf="/etc/squid3/squid.conf" squid_pidfile="/var/run/squid3.pid" squid_port="3128" squid_stop_timeout="30" \
      op start interval="0" timeout="60s" \
      op stop interval="0" timeout="120s" \
      op monitor interval="20s" timeout="30s"
# crm configure primitive HTTPCheck ocf:myscripts:HTTPCheck \
      params url="kb.france.local" \
      op monitor interval="120s"
# crm configure primitive WebSite lsb:apache2 op monitor interval="60sec"
# crm configure location proxy-prefer-node1 Proxy 50: vm-debian1
# crm configure location website-prefer-node1 WebSite 50: vm-debian1
# crm configure colocation proxy-with-clusterip inf: Proxy ClusterIP
# crm configure colocation website-with-clusterip inf: WebSite ClusterIP
# crm configure colocation httpcheck-with-proxy inf: HTTPCheck Proxy
# crm configure order apache-after-clusterip inf: ClusterIP WebSite
# crm configure order squid-after-clusterip inf: ClusterIP Proxy
# crm configure order httpcheck-after-proxy inf: Proxy HTTPCheck