[pgpool-general: 9364] Re: WARNING: failed to lock semaphore

Tatsuo Ishii ishii at postgresql.org
Fri Feb 14 21:11:26 JST 2025 

> Dear Tatsuo,
> 
> Thank you for your continued support.
> 
> I have attached the trace file and the pgpool .log for your review. I don't
> see the semaphore error within the log itself.

Altough the strace file does not include semget call, it seems the
semaphore was actually created thus sr_check process did not emit the
semaphore error. You can make sure that by using ipcs -s command.

> Could you please clarify how
> to identify if pgpool failed to acquire the semaphore at startup?

You should see FATAL message in the log something like:

Unable to create semaphores:8

> Additionally, could you provide guidance on how to resolve this issue if it
> has occurred?

I recommend to try restarting pgpool. Please make sure that before
restart pgpool. shutdown pgpool and make sure there's no pgpool
related process remain.

BTW, this kind of problem is really hard to resolve because I failed
to reproduce the problem. If I were you, I would attach gdb to pgpool
and set break point at pool_semaphore_create(), then use run command
of gdb to see if pgpool actually creates semaphore or not.

Best reagards,
--
Tatsuo Ishii
SRA OSS K.K.
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp


> Best Regards,
> Maria
> 
> 
> On Wed, Feb 12, 2025 at 9:20 PM Tatsuo Ishii <ishii at postgresql.org> wrote:
> 
>> [Cc:ed to pgpoo-general]
>>
>> Hello Maria,
>>
>> Please Cc: to pgpool-general so that our community can share valuable
>> info.
>>
>> > Hello Tatsuo,
>> >
>> > This is how we installed pgpool.
>> >
>> > mkdir -p /local/opt/points/pgpool/4.4.6
>> > cd /usr/src
>> > wget
>> https://protect2.fireeye.com/v1/url?k=31323334-501d2dca-3131568e-454455534531-6acd631f770dce77&q=1&e=50e5f7b9-15c6-4965-86ea-096504160ff4&u=https%3A%2F%2Fwww.pgpool.net%2Fmediawiki%2Fimages%2Fpgpool-II-4.4.6.tar.gz
>> > tar -xvzf pgpool-II-4.4.6.tar.gz
>> > cd pgpool-II-4.4.6/
>> > ./configure --prefix=/local/opt/points/pgpool/4.4.6
>> --enable-sequence-lock
>> > --with-openssl
>> > make ; make install
>> >
>> > Are we missing something?
>>
>> No, except --enable-sequence-lock is not necessary if you run pgpool
>> with streaming replication mode.
>>
>> BTW, I suspect the source of the problem is, pgpool failed to acquire
>> semaphore at the startup for some reasons. To confirm the theory (and
>> possibly to know the cause), can you run pgpool with strace and share
>> the output?  For example:
>>
>> strace -f --trace=ipc pgpool [options....]
>>
>> If you succeed, you will see something like:
>>
>> semget(IPC_PRIVATE, 8, IPC_CREAT|IPC_EXCL|0600) = 655401
>> semctl(655401, 0, SETVAL, 0x7ffe00000001) = 0
>> semctl(655401, 1, SETVAL, 0x7ffe00000001) = 0
>> semctl(655401, 2, SETVAL, 0x7ffe00000001) = 0
>> semctl(655401, 3, SETVAL, 0x7ffe00000001) = 0
>> semctl(655401, 4, SETVAL, 0x7ffe00000001) = 0
>> semctl(655401, 5, SETVAL, 0x7ffe00000001) = 0
>> semctl(655401, 6, SETVAL, 0x7ffe00000001) = 0
>> semctl(655401, 7, SETVAL, 0x7ffe00000001) = 0
>>
>> > On Wed, Feb 12, 2025 at 10:23 AM Maria Nieva <maria.nieva at points.com>
>> wrote:
>> >
>> >>
>> >> Dear Tatsuo,
>> >>
>> >> Thank you for your response.
>> >>
>> >> I've attached the pgpool log from our production environment during the
>> >> upgrade.   The semaphore warning shows right away.  We have since rolled
>> >> back to version 4.1.1. I've also included the config file for your
>> >> reference.
>> >>
>> >> Please let me know if you have any further questions.
>> >>
>> >>
>> >> Best Regards,
>> >> Maria
>> >>
>> >>
>> >>
>> >>
>> >> On Tue, Feb 11, 2025 at 7:40 PM Tatsuo Ishii <ishii at postgresql.org>
>> wrote:
>> >>
>> >>> [Cc:ed to pgpool-general]
>> >>>
>> >>> Thank you for the log. There's no semaphore errors. Now I need to know
>> >>> how you installed 4.4.6. If from RPM, did you use Pgpool-II community
>> >>> version?
>> >>>
>> >>>
>> https://protect2.fireeye.com/v1/url?k=31323334-501d2dca-3131568e-454455534531-ba8a251b4f567a67&q=1&e=e45aec18-0dee-41a2-bde7-d39d532f71f2&u=https%3A%2F%2Fpgpool.net%2Fmediawiki%2Findex.php%2FYum_Repository
>> >>>
>> >>> Can you share pgpool.conf? (please mask sensitive information such as
>> >>> password or IP).
>> >>>
>> >>> Best reagards,
>> >>> --
>> >>> Tatsuo Ishii
>> >>> SRA OSS K.K.
>> >>> English:
>> >>>
>> https://protect2.fireeye.com/v1/url?k=31323334-501d2dca-3131568e-454455534531-bbb96eab86ee7dab&q=1&e=e45aec18-0dee-41a2-bde7-d39d532f71f2&u=http%3A%2F%2Fwww.sraoss.co.jp%2Findex_en%2F
>> >>> Japanese:http://www.sraoss.co.jp
>> >>>
>> >>>
>> >>> > Dear Tatsuo,
>> >>> >
>> >>> > Thank you for your prompt response. I have attached our pgpool log
>> for
>> >>> your
>> >>> > review.
>> >>> >
>> >>> > Please advise on how to resolve the semaphore error. Your guidance is
>> >>> > greatly appreciated.
>> >>> >
>> >>> >
>> >>> >
>> >>> >
>> >>> > On Mon, Feb 10, 2025 at 7:09 PM Tatsuo Ishii <ishii at postgresql.org>
>> >>> wrote:
>> >>> >
>> >>> >> > Hello,
>> >>> >> >
>> >>> >> > We recently upgraded pgpool from 4.1.1 to 4.4.6.  We noticed that
>> we
>> >>> are
>> >>> >> > getting this
>> >>> >> >
>> >>> >> > 2025-02-08 01:39:33.724 PROCESS: WORKER DB: [No Connection] USER:
>> [No
>> >>> >> > Connection] APPL: sr_check_worker % WARNING:  failed to lock
>> >>> semaphore
>> >>> >> [snip]
>> >>> >> > Connection] APPL: sr_check_worker % LOCATION:  pool_sema.c:202
>> >>> >> >
>> >>> >> > This doesn't show much
>> >>> >> >
>> >>> >> > ipcs -s
>> >>> >> >
>> >>> >> > ------ Semaphore Arrays --------
>> >>> >> > key        semid      owner      perms      nsems
>> >>> >>
>> >>> >> Strange. In your case you should see something like:
>> >>> >>
>> >>> >> ------ Semaphore Arrays --------
>> >>> >> key        semid      owner      perms      nsems
>> >>> >> 0x00000000 655386     t-ishii    600        8
>> >>> >>
>> >>> >> ("t-ishii" is the user name of the user who started pgpool)
>> >>> >>
>> >>> >> > This is in our *pgpool.conf *
>> >>> >> > backend_clustering_mode = 'streaming_replication'
>> >>> >> > num_init_children = 250
>> >>> >> > min_spare_children = 25
>> >>> >> > max_spare_children = 50
>> >>> >> > max_pool = 2
>> >>> >> > child_life_time = 300
>> >>> >> > child_max_connections = 2
>> >>> >> > connection_life_time = 300
>> >>> >> > client_idle_limit = 300
>> >>> >> >
>> >>> >> > Can you please help us understand why we see the warning? What is
>> it?
>> >>> >>
>> >>> >> Pgpool worker process complains that it cannot aquire a semaphore.
>> >>> >>
>> >>> >> > How
>> >>> >> > do we resolve this?
>> >>> >>
>> >>> >> To resolve the problem, we need the pgpool log at the start up
>> because
>> >>> >> the semaphore is configured at that time.
>> >>> >>
>> >>> >> Best reagards,
>> >>> >> --
>> >>> >> Tatsuo Ishii
>> >>> >> SRA OSS K.K.
>> >>> >> English:
>> >>> >>
>> >>>
>> https://protect2.fireeye.com/v1/url?k=31323334-501d2dca-3131568e-454455534531-bbb96eab86ee7dab&q=1&e=beae9c64-ec11-4d5c-8783-026e28457a81&u=http%3A%2F%2Fwww.sraoss.co.jp%2Findex_en%2F
>> >>> >> Japanese:http://www.sraoss.co.jp
>> >>> >>
>> >>>
>> >>
>>

More information about the pgpool-general mailing list