[pgpool-general: 1565] Online Recovery failed with unknown reason

[ning chan]

Hi folks,

I have a setup of two nodes with Streaming Replication, pgpool running on
both nodes monitoring each other backends. Everything is working fine,
failover work and oneline recovery work like a charm.

Here is the problem, while server0 failover to server1, and server1 recover
server0 had no problem; however, when i failover back, server1 failover to
server0, server0 is not able to recover server1.

>From the log on Primary:
Apr  1 16:14:52 se032c-94-30 pgpool[2456]: starting recovering node 1
Apr  1 16:14:52 se032c-94-30 pgpool[2456]: starting recovery command:
"SELECT pgpool_recovery('basebackup.sh', 'se032c-94-31',
'/opt/postgres/9.2/data')"
Apr  1 16:14:52 se032c-94-30 postgres[3096]: [2-1]
[unknown]postgres3096ERROR:  pgpool_recovery failed
Apr  1 16:14:52 se032c-94-30 postgres[3096]: [2-2]
[unknown]postgres3096STATEMENT:  SELECT pgpool_recovery('basebackup.sh',
'se032c-94-31', '/opt/postgres/9.2/data')

The exact same script used on both node, so why one work but not the others.
And from the log, I have no idea why it fails, setting debug=[1,2,3] does
not help on troubleshooting.

I then went to manually execute the pgpool_recovery statement, i see the
following:

template1=# SELECT pgpool_recovery('basebackup.sh', 'se032c-94-31',
'/opt/postgres/9.2/data');
ERROR:  pgpool_recovery failed
template1=# SELECT pgpool_recovery('basebackup.sh', '10.89.94.31',
'/opt/postgres/9.2/data');
 pgpool_recovery
-----------------
 t
(1 row)


If hostname is used, the command will failed, but IP works fine.

I double check the hostname is pingable from localhost.

[root at se032c-94-30 ~]# ping se032c-94-31
PING se032c-94-31.cisco.com (10.89.94.31) 56(84) bytes of data.
64 bytes from se032c-94-31.abc.com (10.89.94.31): icmp_seq=1 ttl=64
time=0.254 ms
64 bytes from se032c-94-31.abc.com (10.89.94.31): icmp_seq=2 ttl=64
time=0.311 ms

Please help!

Thanks~
Ning
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.pgpool.net/pipermail/pgpool-general/attachments/20130401/44afbee7/attachment.htm>