<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    Correct.  It's as if pgpool doesn't even see backend_hostname0.  (I
    tried commenting out all of the backend_*1 config items, and pgpool
    didn't see <b>anything</b>. "psql --port=9999" refused connection.)<br>
    <br>
    <font face="monospace">$ psql --port=9999 -c "\x" -c "show
      pool_nodes;"<br>
      Expanded display is on.<br>
      -[ RECORD 1 ]----------+--------------------<br>
      node_id                | 0<br>
      hostname               | FISPCCPGS405a<br>
      port                   | 5432<br>
      status                 | down  
<<<<<<<<<<<<<<<<<<<<br>
      pg_status              | up<br>
      lb_weight              | 0.666667<br>
      role                   | primary<br>
      pg_role                | primary<br>
      select_cnt             | 0<br>
      load_balance_node      | false<br>
      replication_delay      | 0<br>
      replication_state      |<br>
      replication_sync_state |<br>
      last_status_change     | 2023-02-03 23:07:59<br>
      -[ RECORD 2 ]----------+--------------------<br>
      node_id                | 1<br>
      hostname               | FISPCCPGS405b<br>
      port                   | 5432<br>
      status                 | up<br>
      pg_status              | up<br>
      lb_weight              | 0.333333<br>
      role                   | standby<br>
      pg_role                | standby<br>
      select_cnt             | 0<br>
      load_balance_node      | true<br>
      replication_delay      | 0<br>
      replication_state      |<br>
      replication_sync_state |<br>
      last_status_change     | 2023-02-03 23:07:59<br>
    </font><br>
    <br>
    <div class="moz-cite-prefix">On 2/3/23 18:15, Tatsuo Ishii wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:20230204.091528.244601024263051504.t-ishii@sranhm.sra.co.jp">
      <pre class="moz-quote-pre" wrap="">It seems pgpool thinks backend node 0 is down. To confirm this, can
you share pool_status file and the result of show pool_nodes?

</pre>
      <blockquote type="cite">
        <pre class="moz-quote-pre" wrap="">Logs attached, with log_statement = 'all'.

I don't see any attempted connections to the primary server when
pgpool is starting up.

On 2/3/23 03:25, Tatsuo Ishii wrote:
</pre>
        <blockquote type="cite">
          <pre class="moz-quote-pre" wrap="">Can you share PostgreSQL log of the primary with log_statement =
'all'?  I would like to confirm that queries sent from sr_check worker
are reached to the primary. If so, you should see something like:

1771450 2023-02-03 18:19:05.585 JST LOG: statement: SELECT
pg_is_in_recovery()
1771463 2023-02-03 18:19:15.597 JST LOG: statement: SELECT
pg_current_wal_lsn()

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: <a class="moz-txt-link-freetext" href="http://www.sraoss.co.jp/index_en/">http://www.sraoss.co.jp/index_en/</a>
Japanese:<a class="moz-txt-link-freetext" href="http://www.sraoss.co.jp">http://www.sraoss.co.jp</a>

</pre>
          <blockquote type="cite">
            <pre class="moz-quote-pre" wrap="">Attached are three log files (pgpool, the primary and replicated
servers).

The primary is definitely not in replication mode.

On 2/1/23 00:04, Tatsuo Ishii wrote:
</pre>
            <blockquote type="cite">
              <blockquote type="cite">
                <pre class="moz-quote-pre" wrap="">There must have been a miscommunication; I thought I attached my
pgpool.conf and the log file to a previous email, but maybe not.

I fixed the backend_port0 problem last week.
</pre>
              </blockquote>
              <pre class="moz-quote-pre" wrap="">Ok.

</pre>
              <blockquote type="cite">
                <pre class="moz-quote-pre" wrap="">pgppol is already running with pgpool.conf log_min_messages=debug3. Is
that sufficient?
</pre>
              </blockquote>
              <pre class="moz-quote-pre" wrap="">Yes.

</pre>
              <blockquote type="cite">
                <pre class="moz-quote-pre" wrap="">Attached is the error log from when I last started pgpool, and the
pgpool.conf from that time.
</pre>
              </blockquote>
              <pre class="moz-quote-pre" wrap="">I see some errors with streaming replication check process:

2023-01-26 13:31:04.594: sr_check_worker pid 796880: DEBUG: do_query:
extended:0 query:"SELECT pg_current_wal_lsn()"
2023-01-26 13:31:04.594: sr_check_worker pid 796880: CONTEXT: while
checking replication time lag
2023-01-26 13:31:09.594: health_check1 pid 796881: DEBUG: health
check: clearing alarm
2023-01-26 13:31:09.603: health_check1 pid 796881: DEBUG: authenticate
kind = 10
2023-01-26 13:31:09.612: health_check1 pid 796881: DEBUG: SCRAM
authentication successful for user:pool_health_check
2023-01-26 13:31:09.612: health_check1 pid 796881: DEBUG: authenticate
backend: key data received
2023-01-26 13:31:09.612: health_check1 pid 796881: DEBUG: authenticate
backend: transaction state: I
2023-01-26 13:31:09.612: health_check1 pid 796881: DEBUG: health
check: clearing alarm
2023-01-26 13:31:09.612: health_check1 pid 796881: DEBUG: health
check: clearing alarm
2023-01-26 13:31:14.595: sr_check_worker pid 796880: FATAL: Backend
throw an error message
2023-01-26 13:31:14.595: sr_check_worker pid 796880: DETAIL: Exiting
current session because of an error from backend
2023-01-26 13:31:14.595: sr_check_worker pid 796880: HINT: BACKEND
Error: "recovery is in progress"
2023-01-26 13:31:14.595: sr_check_worker pid 796880: CONTEXT: while
checking replication time lag

sr_check_process tried to dtermin WAL LSN on backend0 by issuing
"SELECT pg_current_wal_lsn()" to backend0 but failed with:

</pre>
              <blockquote type="cite">
                <pre class="moz-quote-pre" wrap="">2023-01-26 13:31:14.595: sr_check_worker pid 796880: HINT: BACKEND
Error: "recovery is in progress"
</pre>
              </blockquote>
              <pre class="moz-quote-pre" wrap="">This suggests that backend0 is running as a standby server. I guess
there's something wrong with the setting in backend0.  Maybe
standby.signal exists?  Can you share PostgreSQL log of backend0 at
it's start up?

Best reagards,
--
Tatsuo Ishii
SRA OSS LLC
English: <a class="moz-txt-link-freetext" href="http://www.sraoss.co.jp/index_en/">http://www.sraoss.co.jp/index_en/</a>
Japanese:<a class="moz-txt-link-freetext" href="http://www.sraoss.co.jp">http://www.sraoss.co.jp</a>
</pre>
            </blockquote>
            <pre class="moz-quote-pre" wrap="">-- 
Born in Arizona, moved to Babylonia.
</pre>
          </blockquote>
        </blockquote>
        <pre class="moz-quote-pre" wrap="">
-- 
Born in Arizona, moved to Babylonia.
</pre>
      </blockquote>
    </blockquote>
    <br>
    <div class="moz-signature">-- <br>
      Born in Arizona, moved to Babylonia.</div>
  </body>
</html>