[pgpool-hackers: 2314] Re: Segmentation fault in Pgpool child process

[Tatsuo Ishii]

Usama,

Thank you for letting me know it.  The problem only happens when node
0 is stopped.  The segfault happens while initializing pgpool child
process starting up. The bug was introduced in commit
18f95017586aacb7254bbccda18ba67c01468483 by me, sigh. Also 3.5 stable
tree has the same problem. I have pushed fix to 3.6 and 3.5
stable. More invasive fix was pushed to master branch.

Because the bug is significant, probably we need to release new minor
versions for 3.5 and 3.6.

Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese:http://www.sraoss.co.jp

> Hi Ishii-San
> 
> I am getting the segmentation fault in Pgpool-II child process on master
> and 3_6 stable branches.
> 
> Steps to reproduce.
> 
> 1-) Start the pgpool-II with two backend nodes (both PG up and running).
> 2-) Stop one backend node (bin/pg_ctl -D data_5432/ stop)
> 3-) pgpool-II will perform failover.
> 4-) Connect to Pgpool-II using PSQL after failover is done.
> 
> Step 4 is producing the segmentation fault
> 
> The configuration file I was using for testing is attached and see below
> the stack trace.
> 
> --stack trace--
> 
> [usama at localhost pgpool]$ gdb --core /tmp/core_pgpool.97854 bin/pgpool
> GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-94.el7
> Copyright (C) 2013 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html
>>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-redhat-linux-gnu".
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>...
> Reading symbols from
> /home/usama/work/community/installed/pgpool/bin/pgpool...done.
> [New LWP 97854]
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib64/libthread_db.so.1".
> Core was generated by `pgpool: usama postgres  idle
>                          '.
> Program terminated with signal 11, Segmentation fault.
> #0  0x0000000000422f21 in select_load_balancing_node () at
> protocol/child.c:1669
> 1669 char *database = MASTER_CONNECTION(ses->backend)->sp->database;
> Missing separate debuginfos, use: debuginfo-install
> audit-libs-2.6.5-3.el7.x86_64 cyrus-sasl-lib-2.1.26-20.el7_2.x86_64
> glibc-2.17-157.el7.x86_64 keyutils-libs-1.5.8-3.el7.x86_64
> krb5-libs-1.14.1-27.el7_3.x86_64 libcap-ng-0.7.5-4.el7.x86_64
> libcom_err-1.42.9-9.el7.x86_64 libgcc-4.8.5-11.el7.x86_64
> libmemcached-1.0.16-5.el7.x86_64 libselinux-2.5-6.el7.x86_64
> libstdc++-4.8.5-11.el7.x86_64 nspr-4.13.1-1.0.el7_3.x86_64
> nss-3.28.2-1.6.el7_3.x86_64 nss-softokn-freebl-3.16.2.3-14.4.el7.x86_64
> nss-util-3.28.2-1.1.el7_3.x86_64 openldap-2.4.40-13.el7.x86_64
> openssl-libs-1.0.1e-60.el7_3.1.x86_64 pam-1.1.8-18.el7.x86_64
> pcre-8.32-15.el7_2.1.x86_64 zlib-1.2.7-17.el7.x86_64
> (gdb) bt
> #0  0x0000000000422f21 in select_load_balancing_node () at
> protocol/child.c:1669
> #1  0x000000000043d935 in pool_init_session_context (frontend=<optimized
> out>, backend=<optimized out>) at context/pool_session_context.c:97
> #2  0x0000000000423dee in do_child (fds=fds at entry=0xf41600) at
> protocol/child.c:350
> #3  0x00000000004087e5 in fork_a_child (fds=0xf41600, id=3) at
> main/pgpool_main.c:586
> #4  0x000000000040bf6e in failover () at main/pgpool_main.c:1952
> #5  0x000000000040ccce in sigusr1_interupt_processor () at
> main/pgpool_main.c:1427
> #6  0x000000000040df9a in PgpoolMain (discard_status=discard_status at entry=1
> '\001', clear_memcache_oidmaps=clear_memcache_oidmaps at entry=0 '\000') at
> main/pgpool_main.c:436
> #7  0x0000000000406dbe in main (argc=<optimized out>, argv=<optimized out>)
> at main/main.c:300
> (gdb)
> 
> 
> Thanks
> Best Regards
> Muhammad Usama