[pgpool-hackers: 3451] Re: Pgpool suside if fails to connect to trusted servers

[Siva U]

Hi Tatsuo,

We are also facing the same issue. But network team is claiming no network
issue and said pool have to handle the situation.
We are also in dialama.
Please suggest what is process to follow to overcome this issues? .
What are config parameters to configure to withstand this kind of issues? .

Your suggestions highly appreciated.
Thanks a lot for your support.

Regards,
Siva.

On Friday, September 13, 2019, Tatsuo Ishii <ishii at sraoss.co.jp> wrote:

> In the manual:
>
> 5.14.3. Upstream server connection
>
> ------------------------------------------------------------------
> trusted_servers (string)
>
>     Specifies the list of trusted servers to check the up stream
>     connections. Each server in the list is required to respond to
>     ping. Specify a comma separated list of servers such as
>     "hostA,hostB,hostC". If none of the server are reachable, watchdog
>     will regard it as failure of the Pgpool-II. Therefore, it is
>     recommended to specify multiple servers.
> ------------------------------------------------------------------
>
> It's not clear actually what will happen after "watchdog will regard
> it as failure of the Pgpool-II." What actually happens is, Pgpool-II
> sucides (see the log below for example).
>
> I think we should clearly stat that Pgpool-II will go down if pgpool
> cannot reach to non of trusted servers, something like:
>
> ------------------------------------------------------------------
> trusted_servers (string)
>
>     Specifies the list of trusted servers to check the up stream
>     connections. Each server in the list is required to respond to
>     ping. Specify a comma separated list of servers such as
>     "hostA,hostB,hostC". If none of the server are reachable, watchdog
>     will regard it as failure of the Pgpool-II and the Pgpool-II will
>     shut down. Therefore, it is recommended to specify multiple
>     servers.
> ------------------------------------------------------------------
>
>
> 2019-09-13 10:35:53: pid 30659: WARNING:  watchdog failed to ping
> host"192.192.192.192"
> 2019-09-13 10:35:53: pid 30659: DETAIL:  ping process exits with code: 1
> 2019-09-13 10:35:53: pid 30659: WARNING:  watchdog lifecheck, failed to
> connect to any trusted servers
> 2019-09-13 10:35:53: pid 30659: LOG:  informing the node status change to
> watchdog
> 2019-09-13 10:35:53: pid 30659: DETAIL:  node id :0 status = "NODE DEAD"
> message:"trusted server is unreachable"
> 2019-09-13 10:35:53: pid 30656: LOG:  new IPC connection received
> 2019-09-13 10:35:53: pid 30656: LOG:  received node status change ipc
> message
> 2019-09-13 10:35:53: pid 30656: DETAIL:  trusted server is unreachable
> 2019-09-13 10:35:53: pid 30656: WARNING:  watchdog lifecheck reported, we
> are disconnected from the network
> 2019-09-13 10:35:53: pid 30656: DETAIL:  changing the state to LOST
> 2019-09-13 10:35:53: pid 30656: LOG:  watchdog node state changed from
> [MASTER] to [LOST]
> 2019-09-13 10:35:53: pid 30656: FATAL:  system has lost the network
> 2019-09-13 10:35:53: pid 30656: LOG:  Watchdog is shutting down
> 2019-09-13 10:35:53: pid 30813: LOG:  watchdog: de-escalation started
> 2019-09-13 10:35:53: pid 30646: LOG:  watchdog child process with pid:
> 30656 exits with status 768
> 2019-09-13 10:35:53: pid 30646: FATAL:  watchdog child process exit with
> fatal error. exiting pgpool-II
> 2019-09-13 10:35:53: pid 30814: LOG:  setting the local watchdog node name
> to "localhost:50000 Linux tishii-CFSV7-1"
> 2019-09-13 10:35:53: pid 30814: LOG:  watchdog cluster is configured with
> 2 remote nodes
> 2019-09-13 10:35:53: pid 30814: LOG:  watchdog remote node:0 on
> localhost:50006
> 2019-09-13 10:35:53: pid 30814: LOG:  watchdog remote node:1 on
> localhost:50010
> 2019-09-13 10:35:53: pid 30814: LOG:  interface monitoring is disabled in
> watchdog
> 2019-09-13 10:35:53: pid 30814: LOG:  watchdog node state changed from
> [DEAD] to [LOADING]
> 2019-09-13 10:35:53: pid 30814: LOG:  new outbound connection to
> localhost:50006
> 2019-09-13 10:35:53: pid 30814: LOG:  new outbound connection to
> localhost:50010
> 2019-09-13 10:35:53: pid 30814: LOG:  watchdog node state changed from
> [LOADING] to [INITIALIZING]
> 2019-09-13 10:35:53: pid 30814: LOG:  new watchdog node connection is
> received from "127.0.0.1:60611"
> 2019-09-13 10:35:53: pid 30814: LOG:  new node joined the cluster
> hostname:"localhost" port:50006 pgpool_port:50004
> 2019-09-13 10:35:53: pid 30814: LOG:  new watchdog node connection is
> received from "127.0.0.1:61123"
> 2019-09-13 10:35:53: pid 30814: LOG:  new node joined the cluster
> hostname:"localhost" port:50010 pgpool_port:50008
> 2019-09-13 10:35:53: pid 30814: LOG:  Watchdog is shutting down
> _______________________________________________
> pgpool-hackers mailing list
> pgpool-hackers at pgpool.net
> http://www.pgpool.net/mailman/listinfo/pgpool-hackers
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.sraoss.jp/pipermail/pgpool-hackers/attachments/20191009/7fc47b31/attachment.html>