[pgpool-general: 7571] Re: Resource temporarily unavailable
Tatsuo Ishii
ishii at sraoss.co.jp
Thu May 13 15:18:48 JST 2021
> Hello Tatsuo,
> This is the result:
> Limit Soft Limit Hard Limit Units
> Max cpu time unlimited unlimited seconds
> Max file size unlimited unlimited bytes
> Max data size unlimited unlimited bytes
> Max stack size 8388608 unlimited bytes
> Max core file size 0 unlimited bytes
> Max resident set unlimited unlimited bytes
> Max processes 514179 514179 processes
> Max open files 1024 4096 files
> Max locked memory 65536 65536 bytes
> Max address space unlimited unlimited bytes
> Max file locks unlimited unlimited locks
> Max pending signals 514179 514179 signals
> Max msgqueue size 819200 819200 bytes
> Max nice priority 0 0
> Max realtime priority 0 0
> Max realtime timeout unlimited unlimited us
>
> Pgpool service starts without any problems and creates about 1008 tasks - problem apprears after some time. Today it happened again:
>
> pgs00000161:~ # systemctl status pgpool-II-11.service
> ● pgpool-II-11.service - PGPool-II Middleware Between PostgreSQL Servers And PostgreSQL Database Clients
> Loaded: loaded (/usr/lib/systemd/system/pgpool-II-11.service; disabled; vendor preset: disabled)
> Active: active (running) since Mon 2021-05-10 16:15:04 CEST; 17h ago
> Process: 20750 ExecStop=/usr/pgpool-11/bin/pgpool -f /etc/pgpool-II-11/pgpool.conf -m fast stop (code=killed, signal=INT)
> Main PID: 21255 (pgpool)
> Tasks: 3 (limit: 1024)
> CGroup: /system.slice/pgpool-II-11.service
> ├─21255 /usr/pgpool-11/bin/pgpool -f /etc/pgpool-II-11/pgpool.conf -n -D
> ├─22359 pgpool: health check process(0)
> └─22360 pgpool: health check process(1)
>
> May 10 23:36:38 pgs00000161 pgpool[21255]: [2906-1] 2021-05-10 23:36:38: pid 21255: LOG: fork a new child process with pid: 14222
> May 10 23:36:38 pgs00000161 pgpool[21255]: [2907-1] 2021-05-10 23:36:38: pid 21255: LOG: child process with pid: 7431 exits wi...tus 256
> May 10 23:36:38 pgs00000161 pgpool[21255]: [2908-1] 2021-05-10 23:36:38: pid 21255: LOG: fork a new child process with pid: 14223
> May 10 23:36:38 pgs00000161 pgpool[21255]: [2909-1] 2021-05-10 23:36:38: pid 21255: LOG: child process with pid: 8142 exits wi...tus 256
> May 10 23:36:38 pgs00000161 pgpool[21255]: [2910-1] 2021-05-10 23:36:38: pid 21255: LOG: fork a new child process with pid: 14224
> May 10 23:36:38 pgs00000161 pgpool[21255]: [2911-1] 2021-05-10 23:36:38: pid 21255: LOG: child process with pid: 9498 exits wi...tus 256
> May 10 23:36:38 pgs00000161 pgpool[21255]: [2912-1] 2021-05-10 23:36:38: pid 21255: LOG: fork a new child process with pid: 14225
> May 10 23:36:38 pgs00000161 pgpool[21255]: [2913-1] 2021-05-10 23:36:38: pid 21255: LOG: child process with pid: 4782 exits wi...tus 256
> May 10 23:36:38 pgs00000161 pgpool[21255]: [2914-1] 2021-05-10 23:36:38: pid 21255: FATAL: failed to fork a child
> May 10 23:36:38 pgs00000161 pgpool[21255]: [2914-2] 2021-05-10 23:36:38: pid 21255: DETAIL: system call fork() failed with rea...ailable
> Hint: Some lines were ellipsized, use -l to show in full.
>
> It looks like processes responsible for client connections disappear suddenly and new instances cannot be created but restart of pgpool service helps and it works well again some time.
> I have recommendation to set TasksMax to "infinity" or value much higher than num_init_children*max_pool, but i will be able to verify it during service window in next six hours.
Another possibility is number of file descriptors. On my Lunux box
after startinf 3-node Pgpool-II with 3-node PostgreSQL, I see:
$ cat /proc/sys/fs/file-nr
21700 0 1612809
So 21700 file descriptors out of 1612809 are used. Do you see number
of the used fds are closing to the max file descriptor on your system?
Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese:http://www.sraoss.co.jp
More information about the pgpool-general
mailing list