[pgpool-general: 7573] Re: Resource temporarily unavailable

Tatsuo Ishii ishii at sraoss.co.jp
Thu May 13 17:54:53 JST 2021


> Hello Tatsuo,
> Thank you for your reply. I checked number of file descriptors but it can't be a problem. Below is result:
> 
> pgs00000162:~ # cat /proc/sys/fs/file-nr
> 18080   0       13147086
> 
> As i mentioned in my last message i have changed "TaskMax" value to "infinity" and pgpool seems to be working fine. I started pgpool service two days ago and till now none failure appeared:

Oh, OK. I though your problem wasn't solved yet. Glad to hear that now
your problem has gone.

> pgs00000162:~ # systemctl status pgpool-II-11.service
> ● pgpool-II-11.service - PGPool-II Middleware Between PostgreSQL Servers And PostgreSQL Database Clients
>    Loaded: loaded (/usr/lib/systemd/system/pgpool-II-11.service; disabled; vendor preset: disabled)
>    Active: active (running) since Tue 2021-05-11 10:28:36 CEST; 1 day 23h ago
>  Main PID: 8500 (pgpool)
>     Tasks: 1009
>    CGroup: /system.slice/pgpool-II-11.service
> 
> My linux is Suse Enterprise Server 12 SP3 (trust me, it is not my choice). 
> Best regards and once again thank you for help.
> 
> Andrzej Rozmus
> Starszy Technolog
> Departament Systemów Administracji Rządowej
> Dział Ministerstwo Finansów
> Zespół Projektowy AIS/AES
> Asseco Poland S.A.
> ul. Branickiego 13
> 02-972 Warszawa
> tel. kom. +48 502 270 059
> andrzej.rozmus at asseco.pl
> 
> 
> 
> -----Original Message-----
> From: Tatsuo Ishii <ishii at sraoss.co.jp> 
> Sent: Thursday, May 13, 2021 8:19 AM
> To: Rozmus Andrzej <Andrzej.Rozmus at asseco.pl>
> Cc: pgpool-general at pgpool.net; Wierzbicki Sebastian <sebastian.wierzbicki at asseco.pl>; tomasz.pajda at mf.gov.pl
> Subject: Re: [pgpool-general: 7563] Resource temporarily unavailable
> 
>> Hello Tatsuo,
>> This is the result:
>> Limit                     Soft Limit           Hard Limit           Units
>> Max cpu time              unlimited            unlimited            seconds
>> Max file size             unlimited            unlimited            bytes
>> Max data size             unlimited            unlimited            bytes
>> Max stack size            8388608              unlimited            bytes
>> Max core file size        0                    unlimited            bytes
>> Max resident set          unlimited            unlimited            bytes
>> Max processes             514179               514179               processes
>> Max open files            1024                 4096                 files
>> Max locked memory         65536                65536                bytes
>> Max address space         unlimited            unlimited            bytes
>> Max file locks            unlimited            unlimited            locks
>> Max pending signals       514179               514179               signals
>> Max msgqueue size         819200               819200               bytes
>> Max nice priority         0                    0
>> Max realtime priority     0                    0
>> Max realtime timeout      unlimited            unlimited            us
>> 
>> Pgpool service starts without any problems and creates about 1008 tasks - problem apprears after some time. Today it happened again:
>> 
>> pgs00000161:~ # systemctl status pgpool-II-11.service ● 
>> pgpool-II-11.service - PGPool-II Middleware Between PostgreSQL Servers And PostgreSQL Database Clients
>>    Loaded: loaded (/usr/lib/systemd/system/pgpool-II-11.service; disabled; vendor preset: disabled)
>>    Active: active (running) since Mon 2021-05-10 16:15:04 CEST; 17h ago
>>   Process: 20750 ExecStop=/usr/pgpool-11/bin/pgpool -f 
>> /etc/pgpool-II-11/pgpool.conf -m fast stop (code=killed, signal=INT)  Main PID: 21255 (pgpool)
>>     Tasks: 3 (limit: 1024)
>>    CGroup: /system.slice/pgpool-II-11.service
>>            ├─21255 /usr/pgpool-11/bin/pgpool -f /etc/pgpool-II-11/pgpool.conf -n -D
>>            ├─22359 pgpool: health check process(0)
>>            └─22360 pgpool: health check process(1)
>> 
>> May 10 23:36:38 pgs00000161 pgpool[21255]: [2906-1] 2021-05-10 
>> 23:36:38: pid 21255: LOG:  fork a new child process with pid: 14222 
>> May 10 23:36:38 pgs00000161 pgpool[21255]: [2907-1] 2021-05-10 
>> 23:36:38: pid 21255: LOG:  child process with pid: 7431 exits wi...tus 
>> 256 May 10 23:36:38 pgs00000161 pgpool[21255]: [2908-1] 2021-05-10 
>> 23:36:38: pid 21255: LOG:  fork a new child process with pid: 14223 
>> May 10 23:36:38 pgs00000161 pgpool[21255]: [2909-1] 2021-05-10 
>> 23:36:38: pid 21255: LOG:  child process with pid: 8142 exits wi...tus 
>> 256 May 10 23:36:38 pgs00000161 pgpool[21255]: [2910-1] 2021-05-10 
>> 23:36:38: pid 21255: LOG:  fork a new child process with pid: 14224 
>> May 10 23:36:38 pgs00000161 pgpool[21255]: [2911-1] 2021-05-10 
>> 23:36:38: pid 21255: LOG:  child process with pid: 9498 exits wi...tus 
>> 256 May 10 23:36:38 pgs00000161 pgpool[21255]: [2912-1] 2021-05-10 
>> 23:36:38: pid 21255: LOG:  fork a new child process with pid: 14225 
>> May 10 23:36:38 pgs00000161 pgpool[21255]: [2913-1] 2021-05-10 
>> 23:36:38: pid 21255: LOG:  child process with pid: 4782 exits wi...tus 
>> 256 May 10 23:36:38 pgs00000161 pgpool[21255]: [2914-1] 2021-05-10 
>> 23:36:38: pid 21255: FATAL:  failed to fork a child May 10 23:36:38 
>> pgs00000161 pgpool[21255]: [2914-2] 2021-05-10 23:36:38: pid 21255: 
>> DETAIL:  system call fork() failed with rea...ailable
>> Hint: Some lines were ellipsized, use -l to show in full.
>> 
>> It looks like processes responsible for client connections disappear suddenly and new instances cannot be created but restart of pgpool service helps and it works well again some time.
>> I have recommendation to set TasksMax to "infinity" or value much higher than num_init_children*max_pool, but i will be able to verify it during service window in next six hours.
> 
> Another possibility is number of file descriptors. On my Lunux box after startinf 3-node Pgpool-II with 3-node PostgreSQL, I see:
> 
> $ cat /proc/sys/fs/file-nr
> 21700	0	1612809
> 
> So 21700 file descriptors out of 1612809 are used. Do you see number of the used fds are closing to the max file descriptor on your system?
> 
> Best regards,
> --
> Tatsuo Ishii
> SRA OSS, Inc. Japan
> English: https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.sraoss.co.jp%2Findex_en.php&data=04%7C01%7CAndrzej.Rozmus%40asseco.pl%7C0754b21105be4cfce5b708d915d6fe44%7C88152bdecfa34a5cb981a785c624bb42%7C0%7C0%7C637564835400381362%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=hgfINGc6Awc3WqDuH9JBi1mMYQfZTxQIb2y3lB7F7mo%3D&reserved=0
> Japanese:https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.sraoss.co.jp%2F&data=04%7C01%7CAndrzej.Rozmus%40asseco.pl%7C0754b21105be4cfce5b708d915d6fe44%7C88152bdecfa34a5cb981a785c624bb42%7C0%7C0%7C637564835400381362%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GKoSEnL3H8%2BXlpeDteC352q2LVgc4dr6KGsAbgJbkuA%3D&reserved=0


More information about the pgpool-general mailing list