[pgpool-general: 9123] Re: Another segmentation fault

[Tatsuo Ishii]

> I've attached a sample pgpool.conf. These get generated from a template.
> Things like identifiers will be different, but all settings are identical
> to the one used in the logs. As you can see, both failover_on_backend_error
> and failover_on_backend_shutdown are enabled.

Thank you. That's valuable information to track the code path.

> I've increased the logging as you requested. It might take some time to
> reproduce the errors. These segfaults do not occur very frequently. We see
> about one failure every few days. I'll report back when I managed to
> reproduce them with the new logging settings.

Ok.

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

> Best regards,
> Emond
> 
> Op za 8 jun 2024 om 04:55 schreef Tatsuo Ishii <ishii at sraoss.co.jp>:
> 
>> > Thank you for the info.  This is really weird. By the time when the
>> > execution reach here, backend->slots[2] must be filled like slot[0]
>> > and slot[1] since VALID_BACKEND(2) returns true which means the
>> > connection to backend 2 is valid. I need more time to investigate
>> > this.
>>
>> I am trying to determine the code path when the segfault occurred.  For
>> this purpose can you share pgpool.conf? I especially interested in
>> failover_on_backend_error and failover_on_backend_shutdown settings.
>>
>> Also can you get pgpool.log when segfaults occur with log_min_messages =
>> debug1?
>>
>> >> For the second crash (in pool_connection_pool.c:326), all slots are
>> null:
>> >> (gdb) info locals
>> >> i = 0
>> >> freed = 0
>> >> closetime = 0
>> >> oldestp = 0x7f474f36e168
>> >> ret = 0x0
>> >> info = 0x55b511283469 <opt_sort>
>> >> p = 0x7f474f36e168
>> >>
>> >> (gdb) p *p
>> >> $7 = {
>> >>   info = 0x7f47470d5c08,
>> >>   slots = {0x0 <repeats 128 times>}
>> >> }
>> >>
>> >> (gdb) p p->slots[0]
>> >> $10 = (POOL_CONNECTION_POOL_SLOT *) 0x0
>> >> (gdb) p p->slots[1]
>> >> $11 = (POOL_CONNECTION_POOL_SLOT *) 0x0
>> >> (gdb) p p->slots[2]
>> >> $12 = (POOL_CONNECTION_POOL_SLOT *) 0x0
>> >
>> > This is strange too. How come p->slots[0-2] is 0 if VALID_BACKEND
>> > returns true. I will look into this.
>> >
>> >> I hope this helps.
>> >>
>> >> Best regards,
>> >> Emond
>> >>
>> >> Op di 4 jun 2024 om 04:56 schreef Tatsuo Ishii <ishii at sraoss.co.jp>:
>> >>
>> >>> > No worries. I hope you had a good trip. Last night we triggered the
>> last
>> >>> > crash again. Is there anything we can do to make it easier for you to
>> >>> find
>> >>> > the cause?
>> >>>
>> >>> It would be helpful if you could share some variable values in the
>> >>> core file. Since I don't have the pgpool load module when you got the
>> >>> core, I cannot inspect the variables using the core you provided.
>> >>>
>> >>> > #0  connect_backend (sp=0x55803eb0a6b8, frontend=0x55803eb08768) at
>> >>> > protocol/child.c:1076
>> >>> > #1  0x000055803ce3d02a in get_backend_connection
>> >>> (frontend=0x55803eb08768)
>> >>> > at protocol/child.c:2112
>> >>> > #2  0x000055803ce38fd5 in do_child (fds=0x55803eabea90) at
>> >>> > protocol/child.c:416
>> >>> > #3  0x000055803cdfea4c in fork_a_child (fds=0x55803eabea90, id=13) at
>> >>> > main/pgpool_main.c:863
>> >>> > #4  0x000055803cdfde30 in PgpoolMain (discard_status=0 '\000',
>> >>> > clear_memcache_oidmaps=0 '\000') at main/pgpool_main.c:561
>> >>> > #5  0x000055803cdfb9e6 in main (argc=2, argv=0x7ffc8cdddda8) at
>> >>> > main/main.c:365
>> >>>
>> >>> Around protocol/child.c:1076:
>> >>>                                 /* set DB node id */
>> >>>                                 pool_set_db_node_id(CONNECTION(backend,
>> >>> i), i);
>> >>>
>> >>> I want to see the values in the "backend" struct. Since CONNECTION
>> >>> macro is used here, you have to do something like in a gdb session.
>> >>>
>> >>> p *backend->slots[0]
>> >>> p *backend->slots[1]
>> >>> p *backend->slots[2]
>> >>>
>> >>> Best reagards,
>> >>> --
>> >>> Tatsuo Ishii
>> >>> SRA OSS LLC
>> >>> English: http://www.sraoss.co.jp/index_en/
>> >>> Japanese:http://www.sraoss.co.jp
>> >>>
>> > _______________________________________________
>> > pgpool-general mailing list
>> > pgpool-general at pgpool.net
>> > http://www.pgpool.net/mailman/listinfo/pgpool-general
>>