[pgpool-general: 8510] Re: pgpool shutdown due to [IN NETWORK TROUBLE]

[Muhammad Usama]

On Wed, Nov 9, 2022 at 12:53 AM Steeve Boulanger <sboulanger29 at gmail.com>
wrote:

> Greetings,
>
> We're facing an issue where an ansible playbook does a network reset on
> all interfaces where we have pgpool running, and this seems to cause it to
> shut down:
>
> ***
> 2022-11-08 18:22:39.737: watchdog pid 1323090: DEBUG:  network event
> received
> 2022-11-08 18:22:39.737: watchdog pid 1323090: DETAIL:  deleted = YES Link
> change event = NO
> 2022-11-08 18:22:39.737: watchdog pid 1323090: DEBUG:  STATE MACHINE
> INVOKED WITH EVENT = NETWORK IP IS REMOVED Current State = STANDBY
> 2022-11-08 18:22:39.737: watchdog pid 1323090: DEBUG:  network interface
> lo having flags 65609
> 2022-11-08 18:22:39.737: watchdog pid 1323090: DEBUG:  network interface
> ens192 having flags 4099
> 2022-11-08 18:22:39.737: watchdog pid 1323090: NOTICE:  network interface
> "ens192" link is inactive
> 2022-11-08 18:22:39.737: watchdog pid 1323090: DEBUG:  network interface
> lo having flags 65609
> 2022-11-08 18:22:39.737: watchdog pid 1323090: DEBUG:  network interface
> lo having flags 65609
> 2022-11-08 18:22:39.737: watchdog pid 1323090: DEBUG:  network interface
> ens192 having flags 4099
> 2022-11-08 18:22:39.737: watchdog pid 1323090: NOTICE:  network interface
> "ens192" link is inactive
> 2022-11-08 18:22:39.737: watchdog pid 1323090: WARNING:  network event has
> occurred and all monitored interfaces are down
> 2022-11-08 18:22:39.737: watchdog pid 1323090: DETAIL:  changing the state
> to in network trouble
> 2022-11-08 18:22:39.737: watchdog pid 1323090: LOG:  watchdog node state
> changed from [STANDBY] to [IN NETWORK TROUBLE]
> 2022-11-08 18:22:39.737: watchdog pid 1323090: DEBUG:  STATE MACHINE
> INVOKED WITH EVENT = STATE CHANGED Current State = IN NETWORK TROUBLE
> 2022-11-08 18:22:39.737: watchdog pid 1323090: FATAL:  system has lost the
> network
> 2022-11-08 18:22:39.737: watchdog pid 1323090: LOG:  Watchdog is shutting
> down
> ***
> And since the playbook runs on all pgpool nodes, this is causing a DB
> service outage.
>
> I've looked at the documentation, but I haven't been able to find if there
> was perhaps a timeout parameter, or some other way to avoid pgpool to shut
> down during this short network reset. Any insights would be greatly
> appreciated!
>

If network interface monitoring is enabled, the Pgpool-II node commits
suicide when all (monitored) network interfaces become inactive.
You need to verify the value of the wd_monitoring_interface_list
configuration in pgpool.conf

see wd_monitoring_interfaces_list
<https://www.pgpool.net/docs/43/en/html/runtime-watchdog-config.html#CONFIG-WATCHDOG-LIFECHECK>
 config parameter for more details

Thanks
Best regards
Muhammad Usama


Thanks,
> Steeve
> _______________________________________________
> pgpool-general mailing list
> pgpool-general at pgpool.net
> http://www.pgpool.net/mailman/listinfo/pgpool-general
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.pgpool.net/pipermail/pgpool-general/attachments/20221206/cbe51ab3/attachment.htm>