[pgpool-general: 9322] pgpool setup throws "Unable to acquire JDBC Connection" Errors

[Sbob]

All;


I have setup 2 HOT standby's off a Primary PostgreSQL instance,

then I setup pgpool on another server with a config that points pgpool 
only at the 2 standby's for read only load balancing


The pgpool config is below, which includes a max_pool of 4 (default) and 
num_init_children = 300


The backend hot standby's have max_connections = 2000



########## pgpool config #############

#
# Ansible managed
#
backend_clustering_mode = 'streaming_replication'
listen_addresses = '*'
port = 5432

unix_socket_directories = '/run/pgpool-II'
pcp_socket_dir = '/tmp'

backend_hostname0 = '***********'
backend_port0 = 5432
backend_weight0 = 1
backend_data_directory0 = '/var/lib/pgsql/14/data'
backend_flag0 = 'DISALLOW_TO_FAILOVER'
backend_application_name0 = 'Standby001'

backend_hostname1 = '************'
backend_weight1 = 1
backend_data_directory1 = '/var/lib/pgsql/14/data'
backend_flag1 = 'DISALLOW_TO_FAILOVER'
backend_application_name1 = 'primaryData1'


enable_pool_hba = off
pool_passwd = ''
allow_clear_text_frontend_auth = on
log_destination = 'stderr'
log_connections = off
log_disconnections = off
log_hostname = off
log_statement = off
logging_collector = on
log_directory = '/tmp/log'
log_filename = 'pgpool-%d.log'
log_truncate_on_rotation = on
log_rotation_size = 0
pid_file_name = '/tmp//pgpool.pid'
logdir = '/tmp/log'
load_balance_mode = on
ignore_leading_white_space = on
sr_check_period = 10
sr_check_user = 'postgres'
sr_check_database = 'postgres'
health_check_period = 5
health_check_timeout = 20
health_check_user = 'postgres'
health_check_password = ''
health_check_user0 = 'postgres'
failover_on_backend_shutdown = on

num_init_children = 300



However when the app team runs thier java apps, which include 18 pods 
and each pod can use multiple connections, every pod uses below configs 
(min = 1, max = 50)

so in theory a max of 900 connections.

they keep getting these errors, see below


Thanks in advance for any help



########## java output log ####################

org.springframework.dao.DataAccessResourceFailureException: Unable to 
acquire JDBC Connection; nested exception is 
org.hibernate.exception.JDBCConnectionException: Unable to acquire JDBC 
Connection

  at 
org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:255)

  at 
org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:233)

  at 
org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.translateExceptionIfPossible(AbstractEntityManagerFactoryBean.java:551)

  at 
org.springframework.dao.support.ChainedPersistenceExceptionTranslator.translateExceptionIfPossible(ChainedPersistenceExceptionTranslator.java:61)

  at 
org.springframework.dao.support.DataAccessUtils.translateIfNecessary(DataAccessUtils.java:242)

  at 
org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:152)

  at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)

  at 
org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:145)

  at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)

  at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)

  at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)

  at 
io.opentelemetry.javaagent.instrumentation.spring.data.v1_8.SpringDataInstrumentationModule$RepositoryInterceptor.invoke(SpringDataInstrumentationModule.java:111)

  at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)

  at 
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)

  at com.sun.proxy.$Proxy264.fetchCmsContractIdFromOds(Unknown Source)

  at 
com.rxtransaction.claims.planfinder.service.impl.PriceFileRequestBuildProcessServiceImpl.benefitPackageInfo(PriceFileRequestBuildProcessServiceImpl.java:393)

  at 
com.rxtransaction.claims.planfinder.service.impl.PriceFileRequestBuildProcessServiceImpl.continueProcessRequest(PriceFileRequestBuildProcessServiceImpl.java:206)

  at 
com.rxtransaction.claims.planfinder.service.batch.steps.CustomNdcItemReader.getPriceFileData(CustomNdcItemReader.java:257)

  at com.rxtransaction.