[pgpool-committers: 3942] pgpool: Fix for 0000289: Inconsistent backend state

[Muhammad Usama]

Fix for 0000289: Inconsistent backend state

Pgpool-II syncs the backend node states at the time of startup which works fine
for almost all cases except when the watchdog cluster becomes partitioned
(because of some network problem) and after recovering from it the Pgpool-II
nodes (that are already up and serving) joins back the cluster. At that time the
backend node status among different nodes can become In-sync among the Pgpool-II
nodes, if the backend node status on the newly joined Pgpool-II is different
from the status on existing watchdog cluster nodes.

Now with this commit, every time the Pgpool-II node joins the watchdog cluster
as a standby either at startup or after some problem recovery, it realigns its
backend node statuses with the cluster leader, by fetching the statuses from
the master/coordinator and updating the local statuses of each backend.

The patch also borrows the logic of smart restarting of Pgpool-II children
from the failover() function to minimise the chances of session disconnection
when the backend node statuses are updated from the master/coordinator Pgpool-II.

Branch
------
master

Details
-------
http://git.postgresql.org/gitweb?p=pgpool2.git;a=commitdiff;h=a1ff51eea5a4e58e387ce3888a89bf056ea9f3c3

Modified Files
--------------
src/include/pool.h                     |   1 +
src/include/watchdog/watchdog.h        |   1 +
src/include/watchdog/wd_ipc_commands.h |  15 ++
src/include/watchdog/wd_ipc_defines.h  |  19 +-
src/include/watchdog/wd_json_data.h    |   2 +
src/main/pgpool_main.c                 | 394 ++++++++++++++++++++++++++-------
src/watchdog/watchdog.c                |  90 ++++++--
src/watchdog/wd_commands.c             | 176 ++++++++++++++-
src/watchdog/wd_json_data.c            |  14 +-
9 files changed, 615 insertions(+), 97 deletions(-)