<div dir="ltr">Hi<br><br>I have pushed the above fix in all branches from pgpool-II 3.2 onwards.<br><a href="http://git.postgresql.org/gitweb?p=pgpool2.git;a=commitdiff;h=6afacb1b19603b37e3d005963182258b9f4fca49">http://git.postgresql.org/gitweb?p=pgpool2.git;a=commitdiff;h=6afacb1b19603b37e3d005963182258b9f4fca49</a><div><br></div><div>Thanks again for your help in verifying and testing the fix.</div><div><br></div><div>Kind regards</div><div>Muhammad Usama</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Feb 20, 2016 at 1:03 AM, Muhammad Usama <span dir="ltr"><<a href="mailto:m.usama@gmail.com" target="_blank">m.usama@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi<br>
<br>
Many thanks for the confirmation, the fix needs to go in all branches and I will push it in the morning.<br>
<br>
Regards<br>
Muhammad Usama<br>
<br>
Sent from my iPhone<br>
<div class="HOEnZb"><div class="h5"><br>
> On 19-Feb-2016, at 11:52 PM, Gerhard Wiesinger <<a href="mailto:lists@wiesinger.com">lists@wiesinger.com</a>> wrote:<br>
><br>
> Hello,<br>
><br>
> Can confirm that this patch worked for me (tested the 3.5 patch version), nearly 2 days without any problem. Can you please add it to the git repo and make a new release (3.4, 3.5).<br>
><br>
> Thnx.<br>
><br>
> Ciao,<br>
> Gerhard<br>
><br>
><br>
>> On 16.02.2016 11:44, Muhammad Usama wrote:<br>
>> Hi<br>
>><br>
>> Many thanks for the reply and a good news that you are not getting<br>
>> stuck connection issue after the patch.<br>
>><br>
>> Thanks<br>
>> Best regards<br>
>> Muhammad Usama<br>
>><br>
>><br>
>>> On Fri, Feb 12, 2016 at 9:45 PM, Paweł Ufnalewski <<a href="mailto:archon@foap.com">archon@foap.com</a>> wrote:<br>
>>> Hmm it looks like it's fine now. Right now I only see these in log:<br>
>>><br>
>>> 2016-02-12 17:28:12: pid 8838: LOG: child process with pid: 27299 exits<br>
>>> with status 256<br>
>>> 2016-02-12 17:28:12: pid 8838: LOG: fork a new child process with pid: 6140<br>
>>> 2016-02-12 17:30:42: pid 8838: LOG: child process with pid: 5571 exits with<br>
>>> status 512<br>
>>> 2016-02-12 17:30:42: pid 8838: LOG: fork a new child process with pid: 6720<br>
>>> 2016-02-12 17:30:43: pid 8838: LOG: child process with pid: 4444 exits with<br>
>>> status 512<br>
>>> 2016-02-12 17:30:43: pid 8838: LOG: fork a new child process with pid: 6751<br>
>>> 2016-02-12 17:35:42: pid 8838: LOG: child process with pid: 6140 exits with<br>
>>> status 512<br>
>>> 2016-02-12 17:35:42: pid 8838: LOG: fork a new child process with pid: 7868<br>
>>> 2016-02-12 17:40:42: pid 8838: LOG: child process with pid: 6751 exits with<br>
>>> status 512<br>
>>> 2016-02-12 17:40:42: pid 8838: LOG: fork a new child process with pid: 9018<br>
>>> 2016-02-12 17:40:42: pid 8838: LOG: child process with pid: 6720 exits with<br>
>>> status 512<br>
>>> 2016-02-12 17:40:42: pid 8838: LOG: fork a new child process with pid: 9019<br>
>>><br>
>>> Thank you!<br>
>>><br>
>>> Best regards,<br>
>>> Paweł Ufnalewski<br>
>>> Infrastructure Architect at Foap.com<br>
>>><br>
>>> W dniu 2016-02-09 o 14:02, Muhammad Usama pisze:<br>
>>><br>
>>>> Hi<br>
>>>><br>
>>>> Many thanks for sharing the pgpool.log, The log shared by you does<br>
>>>> contains some error messages "ERROR: unable to to flush data to<br>
>>>> frontend" that have the potential to cause the stuck connection<br>
>>>> Can you please try out the attached patch if it fix the problem. I am<br>
>>>> attaching the patches for both 3_5 and 3_4 branches, please use the<br>
>>>> respective patch as per your setup. Hopefully this should fix the<br>
>>>> stuck issue.<br>
>>>><br>
>>>> Kind regards<br>
>>>> Muhammad Usama<br>
>>>><br>
>>>><br>
>>>><br>
>>>>> On Mon, Feb 8, 2016 at 8:49 PM, Paweł Ufnalewski <<a href="mailto:archon@foap.com">archon@foap.com</a>> wrote:<br>
>>>>> Hi,<br>
>>>>><br>
>>>>> It looks like it hangs in this places (see attachment). Problem is,<br>
>>>>> that<br>
>>>>> developer responsible for app has changed something in code, so<br>
>>>>> connections<br>
>>>>> now closes properly from client side (before I got a lot of these errors:<br>
>>>>><br>
>>>>> 2016-02-08 09:33:39: pid 8472: ERROR: unable to read data from frontend<br>
>>>>> 2016-02-08 09:33:39: pid 8472: DETAIL: EOF encountered with frontend)<br>
>>>>> .<br>
>>>>><br>
>>>>> Best regards,<br>
>>>>> Paweł Ufnalewski<br>
>>>>> Infrastructure Architect at Foap.com<br>
>>>>><br>
>>>>> W dniu 2016-02-08 o 09:00, Muhammad Usama pisze:<br>
>>>>><br>
>>>>> Hi<br>
>>>>><br>
>>>>> Thanks in advance for the help. If you could share the pgpool-II log<br>
>>>>> when the stuck connection happens that would help us in identifiny and<br>
>>>>> rectifing the problem.<br>
>>>>><br>
>>>>> Thanks<br>
>>>>> Best regards<br>
>>>>> Muhammad Usama<br>
>>>>><br>
>>>>><br>
>>>>> On Mon, Feb 8, 2016 at 11:36 AM, Paweł Ufnalewski <<a href="mailto:archon@foap.com">archon@foap.com</a>><br>
>>>>> wrote:<br>
>>>>><br>
>>>>> Hi,<br>
>>>>><br>
>>>>> just to let you know - I'm having the same problem with 3.4.4<br>
>>>>> version<br>
>>>>> (DISCARD ALL appears slower than in 3.4.3 I think, but it still does).<br>
>>>>> How<br>
>>>>> can I help to fix this problem?<br>
>>>>><br>
>>>>> Best regards,<br>
>>>>> Paweł Ufnalewski<br>
>>>>> Infrastructure Architect at Foap.com<br>
>>>>><br>
>>>>> W dniu 2016-02-01 o 08:44, Muhammad Usama pisze:<br>
>>>>><br>
>>>>> Hi Gerhard<br>
>>>>><br>
>>>>> Many thanks for testing and pointing this out. It's unfortunate that you<br>
>>>>> are<br>
>>>>> still getting the stuck connection issue. If it is possible can you<br>
>>>>> please<br>
>>>>> share the pgpool-II log for the time when this stuck connection issue<br>
>>>>> happens. I am more interested in seeing which exact error message that<br>
>>>>> caused the child process to jump to error handler from where the child<br>
>>>>> process proceeded to send the DISCARD ALL to backend and eventually got<br>
>>>>> stuck. Since after many tries we are not able to reproduce this issue, so<br>
>>>>> log would be really helpful in understanding and fixing the problem.<br>
>>>>><br>
>>>>> Best regards<br>
>>>>> Muhammad Usama<br>
>>>>><br>
>>>>><br>
>>>>> On Sun, Jan 31, 2016 at 9:33 PM, Gerhard Wiesinger <<a href="mailto:lists@wiesinger.com">lists@wiesinger.com</a>><br>
>>>>> wrote:<br>
>>>>><br>
>>>>> On 28.01.2016 01:10, Tatsuo Ishii wrote:<br>
>>>>><br>
>>>>> On 21.01.2016 20:52, Muhammad Usama wrote:<br>
>>>>><br>
>>>>> Hi<br>
>>>>><br>
>>>>> I am looking into this issue. and unfortunately like Ishii-San I am<br>
>>>>> also not able to reproduce it. But I found one issue in 3.4 that might<br>
>>>>> cause the problem. Can you please try the attached patch if it solves<br>
>>>>> the problem. Also, if the problem still persists, it would be really<br>
>>>>> helpful if you could share the pgpool-II log.<br>
>>>>><br>
>>>>> I looked at the patch but it includes only logging changes and no<br>
>>>>> functional changes. Therefore I didn't test it. Do you expect and<br>
>>>>> behavioral changes to fix it, and why?<br>
>>>>><br>
>>>>> elog() is not only a logging function, but also it plays very<br>
>>>>> important role including exception handling and error treatments in<br>
>>>>> pgpool-II. If you are familiar with PostgreSQL internals, you may<br>
>>>>> notice it (elog() was imported from PostgreSQL source tree).<br>
>>>>><br>
>>>>> Tried version 3.5.0 where the patch is included. Still not working. See<br>
>>>>> backtrace below.<br>
>>>>><br>
>>>>> Reverting to 3.3.7 which works perfectly.<br>
>>>>><br>
>>>>> Ciao,<br>
>>>>> Gerhard<br>
>>>>><br>
>>>>> (gdb) back<br>
>>>>> #0 0x00007fd87fdb6d43 in __select_nocancel () from /lib64/libc.so.6<br>
>>>>> #1 0x0000564471af16a1 in pool_check_fd (cp=cp@entry=0x564473dfa610) at<br>
>>>>> protocol/pool_process_query.c:635<br>
>>>>> #2 0x0000564471af1976 in pool_check_fd (cp=cp@entry=0x564473dfa610) at<br>
>>>>> protocol/pool_process_query.c:657<br>
>>>>> #3 0x0000564471b1f67b in pool_read (cp=0x564473dfa610,<br>
>>>>> buf=buf@entry=0x7ffc1d71bf97, len=len@entry=1) at utils/pool_stream.c:162<br>
>>>>> #4 0x0000564471af8e6e in read_kind_from_backend<br>
>>>>> (frontend=frontend@entry=0x564473df3e60,<br>
>>>>> backend=backend@entry=0x564473df2e00,<br>
>>>>> decided_kind=decided_kind@entry=0x7ffc1d71c397 "E") at<br>
>>>>> protocol/pool_process_query.c:3234<br>
>>>>> #5 0x0000564471affdc9 in ProcessBackendResponse<br>
>>>>> (frontend=frontend@entry=0x564473df3e60,<br>
>>>>> backend=backend@entry=0x564473df2e00, state=state@entry=0x7ffc1d71c41c,<br>
>>>>> num_fields=num_fields@entry=0x7ffc1d71c41a) at<br>
>>>>> protocol/pool_proto_modules.c:2356<br>
>>>>> #6 0x0000564471af5b15 in pool_process_query (frontend=0x564473df3e60,<br>
>>>>> backend=0x564473df2e00, reset_request=reset_request@entry=1) at<br>
>>>>> protocol/pool_process_query.c:302<br>
>>>>> #7 0x0000564471aed98c in backend_cleanup (backend=<optimized out>,<br>
>>>>> frontend_invalid=frontend_invalid@entry=0 '\000', frontend=0x564471e09e40<br>
>>>>> <child_frontend>)<br>
>>>>> at protocol/child.c:437<br>
>>>>> #8 0x0000564471af0637 in do_child (fds=fds@entry=0x564473dee030) at<br>
>>>>> protocol/child.c:234<br>
>>>>> #9 0x0000564471ace107 in fork_a_child (fds=0x564473dee030, id=8) at<br>
>>>>> main/pgpool_main.c:678<br>
>>>>> #10 0x0000564471aceb6d in reaper () at main/pgpool_main.c:2254<br>
>>>>> #11 0x0000564471ad322b in PgpoolMain (discard_status=<optimized out>,<br>
>>>>> clear_memcache_oidmaps=<optimized out>) at main/pgpool_main.c:429<br>
>>>>> #12 0x0000564471acc7b1 in main (argc=<optimized out>,<br>
>>>>> argv=0x7ffc1d7219e8)<br>
>>>>> at main/main.c:310<br>
>>>>><br>
>>>>> #1 0x0000564471af16a1 in pool_check_fd (cp=cp@entry=0x564473dfa610) at<br>
>>>>> protocol/pool_process_query.c:635<br>
>>>>> 635 fds = select(fd+1, &readmask, NULL, &exceptmask,<br>
>>>>> timeoutp);<br>
>>>>><br>
>>>>> (gdb) print fd<br>
>>>>> $1 = 8<br>
>>>>> (gdb) print readmask<br>
>>>>> $2 = {fds_bits = {256, 0 <repeats 15 times>}}<br>
>>>>> (gdb) print exceptmask<br>
>>>>> $3 = {fds_bits = {256, 0 <repeats 15 times>}}<br>
>>>>> (gdb) print timeoutp<br>
>>>>> $4 = (struct timeval *) 0x0<br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>> _______________________________________________<br>
>>>>> pgpool-general mailing list<br>
>>>>> <a href="mailto:pgpool-general@pgpool.net">pgpool-general@pgpool.net</a><br>
>>>>> <a href="http://www.pgpool.net/mailman/listinfo/pgpool-general" rel="noreferrer" target="_blank">http://www.pgpool.net/mailman/listinfo/pgpool-general</a><br>
>> _______________________________________________<br>
>> pgpool-general mailing list<br>
>> <a href="mailto:pgpool-general@pgpool.net">pgpool-general@pgpool.net</a><br>
>> <a href="http://www.pgpool.net/mailman/listinfo/pgpool-general" rel="noreferrer" target="_blank">http://www.pgpool.net/mailman/listinfo/pgpool-general</a><br>
><br>
</div></div></blockquote></div><br></div>