[pgpool-general: 9413] Re: FATAL: failed to read kind from backend

[Tatsuo Ishii]

> On Thu, Mar 27, 2025 at 9:28 AM Tatsuo Ishii <ishii at postgresql.org> wrote:
> 
>> > Last week, I disabled load balancing, but the replica still has some
>> weight
>> > Might that be causing the problem?
>> > load_balance_mode = off
>> > backend_weight0 = 2
>> > backend_weight1 = 1
>>
>> No, I don't think so. If load_balance_mode = off, then backend_weight
>> is ignored.
> 
> 
> I'm still getting the WARNING and FATAL messages even after stopping pgpool
> on all three nodes, but only starting pgpool on nodes C and A.

The error (kind mismatch) is nothing related to watchdog.

> pcp_watchdog_info thinks that node B is DEAD.
> 
> [postgres at FISPCDSPGS302c ~]$ pcp_watchdog_info -wv
> Watchdog Cluster Information
> Total Nodes              : 3
> Remote Nodes             : 2
> Member Remote Nodes      : 2
> Alive Remote Nodes       : 1
> Nodes required for quorum: 2
> Quorum state             : QUORUM EXIST
> Local node escalation    : YES
> Leader Node Name         : FISPCDSPGS302c:9999 Linux FISPCDSPGS302c
> Leader Host Name         : FISPCDSPGS302c
> 
> Watchdog Node Information
> Node Name         : FISPCDSPGS302c:9999 Linux FISPCDSPGS302c
> Host Name         : FISPCDSPGS302c
> Delegate IP       : 10.109.160.20
> Pgpool port       : 9999
> Watchdog port     : 9000
> Node priority     : 1
> Status            : 4
> Status Name       : LEADER
> Membership Status : MEMBER
> 
> Node Name         : FISPCDSPGS302a:9999 Linux FISPCDSPGS302a
> Host Name         : FISPCDSPGS302a
> Delegate IP       : 10.109.160.20
> Pgpool port       : 9999
> Watchdog port     : 9000
> Node priority     : 1
> Status            : 7
> Status Name       : STANDBY
> Membership Status : MEMBER
> 
> Node Name         : Not_Set
> Host Name         : FISPCDSPGS302b
> Delegate IP       : Not_Set
> Pgpool port       : 9999
> Watchdog port     : 9000
> Node priority     : 0
> Status            : 0
> Status Name       : DEAD
> Membership Status : MEMBER
> 
> [postgres at FISPCDSPGS302c ~]$ pcp_health_check_stats -vw -n0
> Node Id                       : 0
> Host Name                     : FISPCDSPGS302a
> Port                          : 5432
> Status                        : up
> Role                          : primary
> Last Status Change            : 2025-03-27 08:49:59
> Total Count                   : 766
> Success Count                 : 766
> Fail Count                    : 0
> Skip Count                    : 0
> Retry Count                   : 0
> Average Retry Count           : 0.000000
> Max Retry Count               : 0
> Max Health Check Duration     : 17
> Minimum Health Check Duration : 10
> Average Health Check Duration : 11.481723
> Last Health Check             : 2025-03-27 09:53:14
> Last Successful Health Check  : 2025-03-27 09:53:14
> Last Skip Health Check        :
> Last Failed Health Check      :
> 
> [postgres at FISPCDSPGS302c ~]$ pcp_health_check_stats -vw -n1
> Node Id                       : 1
> Host Name                     : FISPCDSPGS302b
> Port                          : 5432
> Status                        : up
> Role                          : standby
> Last Status Change            : 2025-03-27 08:49:59
> Total Count                   : 768
> Success Count                 : 768
> Fail Count                    : 0
> Skip Count                    : 0
> Retry Count                   : 0
> Average Retry Count           : 0.000000
> Max Retry Count               : 0
> Max Health Check Duration     : 16
> Minimum Health Check Duration : 10
> Average Health Check Duration : 11.330729
> Last Health Check             : 2025-03-27 09:53:24
> Last Successful Health Check  : 2025-03-27 09:53:24
> Last Skip Health Check        :
> Last Failed Health Check      :
> 
> 
> -- 
> Death to <Redacted>, and butter sauce.
> Don't boil me, I'm still alive.
> <Redacted> lobster!