[pgpool-general: 7572] Re: Resource temporarily unavailable

Rozmus Andrzej Andrzej.Rozmus at asseco.pl
Thu May 13 17:06:45 JST 2021


Hello Tatsuo,
Thank you for your reply. I checked number of file descriptors but it can't be a problem. Below is result:

pgs00000162:~ # cat /proc/sys/fs/file-nr
18080   0       13147086

As i mentioned in my last message i have changed "TaskMax" value to "infinity" and pgpool seems to be working fine. I started pgpool service two days ago and till now none failure appeared:

pgs00000162:~ # systemctl status pgpool-II-11.service
● pgpool-II-11.service - PGPool-II Middleware Between PostgreSQL Servers And PostgreSQL Database Clients
   Loaded: loaded (/usr/lib/systemd/system/pgpool-II-11.service; disabled; vendor preset: disabled)
   Active: active (running) since Tue 2021-05-11 10:28:36 CEST; 1 day 23h ago
 Main PID: 8500 (pgpool)
    Tasks: 1009
   CGroup: /system.slice/pgpool-II-11.service

My linux is Suse Enterprise Server 12 SP3 (trust me, it is not my choice). 
Best regards and once again thank you for help.

Andrzej Rozmus
Starszy Technolog
Departament Systemów Administracji Rządowej
Dział Ministerstwo Finansów
Zespół Projektowy AIS/AES
Asseco Poland S.A.
ul. Branickiego 13
02-972 Warszawa
tel. kom. +48 502 270 059
andrzej.rozmus at asseco.pl



-----Original Message-----
From: Tatsuo Ishii <ishii at sraoss.co.jp> 
Sent: Thursday, May 13, 2021 8:19 AM
To: Rozmus Andrzej <Andrzej.Rozmus at asseco.pl>
Cc: pgpool-general at pgpool.net; Wierzbicki Sebastian <sebastian.wierzbicki at asseco.pl>; tomasz.pajda at mf.gov.pl
Subject: Re: [pgpool-general: 7563] Resource temporarily unavailable

> Hello Tatsuo,
> This is the result:
> Limit                     Soft Limit           Hard Limit           Units
> Max cpu time              unlimited            unlimited            seconds
> Max file size             unlimited            unlimited            bytes
> Max data size             unlimited            unlimited            bytes
> Max stack size            8388608              unlimited            bytes
> Max core file size        0                    unlimited            bytes
> Max resident set          unlimited            unlimited            bytes
> Max processes             514179               514179               processes
> Max open files            1024                 4096                 files
> Max locked memory         65536                65536                bytes
> Max address space         unlimited            unlimited            bytes
> Max file locks            unlimited            unlimited            locks
> Max pending signals       514179               514179               signals
> Max msgqueue size         819200               819200               bytes
> Max nice priority         0                    0
> Max realtime priority     0                    0
> Max realtime timeout      unlimited            unlimited            us
> 
> Pgpool service starts without any problems and creates about 1008 tasks - problem apprears after some time. Today it happened again:
> 
> pgs00000161:~ # systemctl status pgpool-II-11.service ● 
> pgpool-II-11.service - PGPool-II Middleware Between PostgreSQL Servers And PostgreSQL Database Clients
>    Loaded: loaded (/usr/lib/systemd/system/pgpool-II-11.service; disabled; vendor preset: disabled)
>    Active: active (running) since Mon 2021-05-10 16:15:04 CEST; 17h ago
>   Process: 20750 ExecStop=/usr/pgpool-11/bin/pgpool -f 
> /etc/pgpool-II-11/pgpool.conf -m fast stop (code=killed, signal=INT)  Main PID: 21255 (pgpool)
>     Tasks: 3 (limit: 1024)
>    CGroup: /system.slice/pgpool-II-11.service
>            ├─21255 /usr/pgpool-11/bin/pgpool -f /etc/pgpool-II-11/pgpool.conf -n -D
>            ├─22359 pgpool: health check process(0)
>            └─22360 pgpool: health check process(1)
> 
> May 10 23:36:38 pgs00000161 pgpool[21255]: [2906-1] 2021-05-10 
> 23:36:38: pid 21255: LOG:  fork a new child process with pid: 14222 
> May 10 23:36:38 pgs00000161 pgpool[21255]: [2907-1] 2021-05-10 
> 23:36:38: pid 21255: LOG:  child process with pid: 7431 exits wi...tus 
> 256 May 10 23:36:38 pgs00000161 pgpool[21255]: [2908-1] 2021-05-10 
> 23:36:38: pid 21255: LOG:  fork a new child process with pid: 14223 
> May 10 23:36:38 pgs00000161 pgpool[21255]: [2909-1] 2021-05-10 
> 23:36:38: pid 21255: LOG:  child process with pid: 8142 exits wi...tus 
> 256 May 10 23:36:38 pgs00000161 pgpool[21255]: [2910-1] 2021-05-10 
> 23:36:38: pid 21255: LOG:  fork a new child process with pid: 14224 
> May 10 23:36:38 pgs00000161 pgpool[21255]: [2911-1] 2021-05-10 
> 23:36:38: pid 21255: LOG:  child process with pid: 9498 exits wi...tus 
> 256 May 10 23:36:38 pgs00000161 pgpool[21255]: [2912-1] 2021-05-10 
> 23:36:38: pid 21255: LOG:  fork a new child process with pid: 14225 
> May 10 23:36:38 pgs00000161 pgpool[21255]: [2913-1] 2021-05-10 
> 23:36:38: pid 21255: LOG:  child process with pid: 4782 exits wi...tus 
> 256 May 10 23:36:38 pgs00000161 pgpool[21255]: [2914-1] 2021-05-10 
> 23:36:38: pid 21255: FATAL:  failed to fork a child May 10 23:36:38 
> pgs00000161 pgpool[21255]: [2914-2] 2021-05-10 23:36:38: pid 21255: 
> DETAIL:  system call fork() failed with rea...ailable
> Hint: Some lines were ellipsized, use -l to show in full.
> 
> It looks like processes responsible for client connections disappear suddenly and new instances cannot be created but restart of pgpool service helps and it works well again some time.
> I have recommendation to set TasksMax to "infinity" or value much higher than num_init_children*max_pool, but i will be able to verify it during service window in next six hours.

Another possibility is number of file descriptors. On my Lunux box after startinf 3-node Pgpool-II with 3-node PostgreSQL, I see:

$ cat /proc/sys/fs/file-nr
21700	0	1612809

So 21700 file descriptors out of 1612809 are used. Do you see number of the used fds are closing to the max file descriptor on your system?

Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.sraoss.co.jp%2Findex_en.php&data=04%7C01%7CAndrzej.Rozmus%40asseco.pl%7C0754b21105be4cfce5b708d915d6fe44%7C88152bdecfa34a5cb981a785c624bb42%7C0%7C0%7C637564835400381362%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=hgfINGc6Awc3WqDuH9JBi1mMYQfZTxQIb2y3lB7F7mo%3D&reserved=0
Japanese:https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.sraoss.co.jp%2F&data=04%7C01%7CAndrzej.Rozmus%40asseco.pl%7C0754b21105be4cfce5b708d915d6fe44%7C88152bdecfa34a5cb981a785c624bb42%7C0%7C0%7C637564835400381362%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GKoSEnL3H8%2BXlpeDteC352q2LVgc4dr6KGsAbgJbkuA%3D&reserved=0


More information about the pgpool-general mailing list