7.4. AWS Configuration Example

This tutorial explains the simple way to try "Watchdog" on AWS and using the Elastic IP Address as the Virtual IP for the high availability solution.

Note: You can use watchdog with Pgpool-II in any mode: replication mode, master/slave mode and raw mode.

7.4.1. AWS Setup

For this example, we will use two node Pgpool-II watchdog cluster. So we will set up two Linux Amazon EC2 instances and one Elastic IP address. So for this example, do the following steps:

7.4.2. Pgpool-II configurations

Mostly the Pgpool-II configurations for this example will be same as in the Section 7.2, except the delegate_IP which we will not set in this example instead we will use wd_escalation_command and wd_de_escalation_command to switch the Elastic IP address to the master/Active Pgpool-II node.

7.4.2.1. Pgpool-II configurations on Instance-1

       use_watchdog = on
       delegate_IP = ''
       wd_hostname = 'instance-1-private-ip'
       other_pgpool_hostname0 = 'instance-2-private-ip'
       other_pgpool_port0 = 9999
       other_wd_port0 = 9000
       wd_escalation_command = '$path_to_script/aws-escalation.sh'
       wd_de_escalation_command = '$path_to_script/aws-de-escalation.sh'
      

7.4.2.2. Pgpool-II configurations on Instance-2

       use_watchdog = on
       delegate_IP = ''
       wd_hostname = 'instance-2-private-ip'
       other_pgpool_hostname0 = 'instance-1-private-ip'
       other_pgpool_port0 = 9999
       other_wd_port0 = 9000
       wd_escalation_command = '$path_to_script/aws-escalation.sh'
       wd_de_escalation_command = '$path_to_script/aws-de-escalation.sh'
      

7.4.3. escalation and de-escalation Scripts

Create the aws-escalation.sh and aws-de-escalation.sh scripts on both instances and point the wd_escalation_command and wd_de_escalation_command to the respective scripts.

Note: You may need to configure the AWS CLI first on all AWS instances to enable the execution of commands used by wd-escalation.sh and wd-de-escalation.sh. See configure AWS CLI

7.4.3.1. escalation script

This script will be executed by the watchdog to assign the Elastic IP on the instance when the watchdog becomes the active/master node. Change the INSTANCE_ID and ELASTIC_IP values as per your AWS setup values.

aws-escalation.sh:

       #! /bin/sh

       ELASTIC_IP=35.163.178.3
       # replace it with the Elastic IP address you
       # allocated from the aws console
       INSTANCE_ID=i-0a9b64e449b17ed4b
       # replace it with the instance id of the Instance
       # this script is installed on

       echo "Assigning Elastic IP $ELASTIC_IP to the instance $INSTANCE_ID"
       # bring up the Elastic IP
       aws ec2 associate-address --instance-id $INSTANCE_ID --public-ip $ELASTIC_IP

       exit 0
      

7.4.3.2. de-escalation script

This script will be executed by watchdog to remove the Elastic IP from the instance when the watchdog resign from the active/master node.

aws-de-escalation.sh:

       #! /bin/sh

       ELASTIC_IP=35.163.178.3
       # replace it with the Elastic IP address you
       # allocated from the aws console

       echo "disassociating the Elastic IP $ELASTIC_IP from the instance"
       # bring down the Elastic IP
       aws ec2 disassociate-address --public-ip $ELASTIC_IP
       exit 0
      

AWS Command References

"Configure AWS CLI", AWS Documentation: Configuring the AWS Command Line Interface.

"associate-address", AWS Documentation: associate-address reference.

"disassociate-address", AWS Documentation: disassociate-address reference.

7.4.4. Try it out

Start Pgpool-II on each server with "-n" switch and redirect log messages to the pgpool.log file. The log message of master/active Pgpool-II node will show the message of Elastic IP assignment.

      LOG:  I am the cluster leader node. Starting escalation process
      LOG:  escalation process started with PID:23543
      LOG:  watchdog: escalation started
             Assigning Elastic IP 35.163.178.3 to the instance i-0a9b64e449b17ed4b
       {
       "AssociationId": "eipassoc-39853c42"
       }
      
      LOG:  watchdog escalation successful
      LOG:  watchdog escalation process with pid: 23543 exit with SUCCESS.
     

Confirm to ping to the Elastic IP address.

      [user@someserver]$ ping 35.163.178.3
      PING 35.163.178.3 (35.163.178.3) 56(84) bytes of data.
      64 bytes from 35.163.178.3: icmp_seq=1 ttl=64 time=0.328 ms
      64 bytes from 35.163.178.3: icmp_seq=2 ttl=64 time=0.264 ms
      64 bytes from 35.163.178.3: icmp_seq=3 ttl=64 time=0.412 ms
     

Try to connect PostgreSQL by "psql -h ELASTIC_IP -p port".

      [user@someserver]$ psql -h 35.163.178.3 -p 9999 -l
     

7.4.5. Switching Elastic IP

To confirm if the Standby server acquires the Elastic IP when the Active server becomes unavailable, Stop the Pgpool-II on the Active server. Then, the Standby server should start using the Elastic IP address, And the Pgpool-II log will show the below messages.

             LOG:  remote node "172.31.2.94:9999 [Linux ip-172-31-2-94]" is shutting down
       LOG:  watchdog cluster has lost the coordinator node
      
      LOG:  watchdog node state changed from [STANDBY] to [JOINING]
      LOG:  watchdog node state changed from [JOINING] to [INITIALIZING]
      LOG:  I am the only alive node in the watchdog cluster
      HINT:  skipping stand for coordinator state
      LOG:  watchdog node state changed from [INITIALIZING] to [MASTER]
      LOG:  I am announcing my self as master/coordinator watchdog node
      LOG:  I am the cluster leader node
      DETAIL:  our declare coordinator message is accepted by all nodes
      LOG:  I am the cluster leader node. Starting escalation process
      LOG:  escalation process started with PID:23543
      LOG:  watchdog: escalation started
             Assigning Elastic IP 35.163.178.3 to the instance i-0dd3e60734a6ebe14
       {
       "AssociationId": "eipassoc-39853c42"
       }
      
      LOG:  watchdog escalation successful
      LOG:  watchdog escalation process with pid: 61581 exit with SUCCESS.
     

Confirm to ping to the Elastic IP address again.

      [user@someserver]$ ping 35.163.178.3
      PING 35.163.178.3 (35.163.178.3) 56(84) bytes of data.
      64 bytes from 35.163.178.3: icmp_seq=1 ttl=64 time=0.328 ms
      64 bytes from 35.163.178.3: icmp_seq=2 ttl=64 time=0.264 ms
      64 bytes from 35.163.178.3: icmp_seq=3 ttl=64 time=0.412 ms
     

Try to connect PostgreSQL by "psql -h ELASTIC_IP -p port".

      [user@someserver]$ psql -h 35.163.178.3 -p 9999 -l