What role does lkv-srvapp39 serve? Is it your postgresql database server? That of a second Uyuni postgresql server you are trying to take data from to merge into a HUB reporting DB? It looks
like your application server may be failing to connect to it. There could be a whole bunch of things that could be causing that:
a) DNS resolution? Maybe the lkv-srvapp39 server doesn’t have a proper DNS entry. Maybe the Uyuni server has the wrong DNS server configured for resolving that name. Can you use nslookup to resolve the name from
the Uyuni app server?
b) typo on the name in the configuration file?
c) missing domain name (not FQDN) and no DNS domain search configured in your resolver configuration? Or it’s a different internal domain from your HUB server and there’s no trust/forwarding setup for DNS between those domains?
d) routing issue?
e) host-based firewall on the (remote?) PG server not allowing connections on the TCP/5432 port?
f) physical firewall or router ACLs preventing the connection?
You’re going to have to identify any other possibilities for your topology and then test all of them using other means (nslookup, ping, traceroute, psql, etc) to see what’s not working.
From: Crefeld, Tobias LKV Bayern e.V. <Tobias.Crefeld@lkv.bayern.de>
Sent: Tuesday, August 2, 2022 3:29 AM
To: General discussion related to the openSUSE Uyuni project <users@lists.uyuni-project.org>
Subject: AW: Subtask mgr-update-reporting failed.
Hi Simon,
thanks for your hints! I followed them and received a lot of messages but actually I’m no java developer.
I tried to reduce the log output by leaving out the “at”-lines but it is still large. Hope that no one gets annoyed too much:
2022-08-02 09:56:00,022 [DefaultQuartzScheduler_Worker-18] WARN org.hibernate.orm.deprecation - HHH90000022: Hibernate's legacy org.hibernate.Criteria API is deprecated; use the JPA javax.persistence.criteria.CriteriaQuery
instead
2022-08-02 09:56:02,718 [DefaultQuartzScheduler_Worker-1] WARN org.hibernate.orm.deprecation - HHH020100: The Ehcache second-level cache provider for Hibernate is deprecated. See
https://hibernate.atlassian.net/browse/HHH-12441 for details.
2022-08-02 09:56:33,348 [C3P0PooledConnectionPoolManager[identityToken->z8kfluaqp0t6ro20pjuq|3ff5aba]-HelperThread-#0] WARN com.mchange.v2.resourcepool.BasicResourcePool -
com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@1c8f4b1f -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition
attempts (30). Last acquisition attempt exception:
org.postgresql.util.PSQLException: The hostname lkv-srvapp39 could not be verified by hostnameverifier PgjdbcHostnameVerifier.
2022-08-02 09:56:33,348 [C3P0PooledConnectionPoolManager[identityToken->z8kfluaqp0t6ro20pjuq|3ff5aba]-HelperThread-#0] WARN com.mchange.v2.resourcepool.BasicResourcePool - Having failed to acquire
a resource,
com.mchange.v2.resourcepool.BasicResourcePool@411ce9ec is interrupting all Threads waiting on a resource to check out. Will try again in response to new client requests.
2022-08-02 09:56:33,348 [DefaultQuartzScheduler_Worker-1] WARN org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator - HHH000342: Could not obtain connection to query metadata : Connections
could not be acquired from the underlying database!
2022-08-02 09:56:33,354 [C3P0PooledConnectionPoolManager[identityToken->z8kfluaqp0t6ro20pjuq|3ff5aba]-HelperThread-#1] WARN com.mchange.v2.resourcepool.BasicResourcePool -
com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@5ab931dd -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition
attempts (30). Last acquisition attempt exception:
org.postgresql.util.PSQLException: The hostname lkv-srvapp39 could not be verified by hostnameverifier PgjdbcHostnameVerifier.
2022-08-02 09:56:33,354 [C3P0PooledConnectionPoolManager[identityToken->z8kfluaqp0t6ro20pjuq|3ff5aba]-HelperThread-#1] WARN com.mchange.v2.resourcepool.BasicResourcePool - Having failed to acquire
a resource,
com.mchange.v2.resourcepool.BasicResourcePool@411ce9ec is interrupting all Threads waiting on a resource to check out. Will try again in response to new client requests.
2022-08-02 09:56:33,360 [DefaultQuartzScheduler_Worker-1] WARN net.sf.ehcache.DiskStorePathManager - diskStorePath '/tmp' is already used by an existing CacheManager either in the same VM or in
a different process.
The diskStore path for this CacheManager will be set to /tmp/ehcache_auto_created2394383558487899130diskstore.
To avoid this warning consider using the CacheManager factory methods to create a singleton CacheManager or specifying a separate ehcache configuration (ehcache.xml) for each CacheManager instance.
2022-08-02 09:56:33,361 [C3P0PooledConnectionPoolManager[identityToken->z8kfluaqp0t6ro20pjuq|3ff5aba]-HelperThread-#2] WARN com.mchange.v2.resourcepool.BasicResourcePool -
com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@68e07fff -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition
attempts (30). Last acquisition attempt exception:
org.postgresql.util.PSQLException: The hostname lkv-srvapp39 could not be verified by hostnameverifier PgjdbcHostnameVerifier.
2022-08-02 09:56:33,362 [C3P0PooledConnectionPoolManager[identityToken->z8kfluaqp0t6ro20pjuq|3ff5aba]-HelperThread-#2] WARN com.mchange.v2.resourcepool.BasicResourcePool - Having failed to acquire
a resource,
com.mchange.v2.resourcepool.BasicResourcePool@411ce9ec is interrupting all Threads waiting on a resource to check out. Will try again in response to new client requests.
2022-08-02 09:56:33,374 [C3P0PooledConnectionPoolManager[identityToken->z8kfluaqp0t6ro20pjuq|3ff5aba]-HelperThread-#1] WARN com.mchange.v2.resourcepool.BasicResourcePool -
com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@3b518b25 -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition
attempts (30). Last acquisition attempt exception:
org.postgresql.util.PSQLException: The hostname lkv-srvapp39 could not be verified by hostnameverifier PgjdbcHostnameVerifier.
2022-08-02 09:56:33,374 [C3P0PooledConnectionPoolManager[identityToken->z8kfluaqp0t6ro20pjuq|3ff5aba]-HelperThread-#0] WARN com.mchange.v2.resourcepool.BasicResourcePool -
com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@5599538e -- Acquisition Attempt Failed!!! Clearing pending acquires. While trying to acquire a needed new resource, we failed to succeed more than the maximum number of allowed acquisition
attempts (30). Last acquisition attempt exception:
org.postgresql.util.PSQLException: The hostname lkv-srvapp39 could not be verified by hostnameverifier PgjdbcHostnameVerifier.
2022-08-02 09:56:33,374 [C3P0PooledConnectionPoolManager[identityToken->z8kfluaqp0t6ro20pjuq|3ff5aba]-HelperThread-#1] WARN com.mchange.v2.resourcepool.BasicResourcePool - Having failed to acquire
a resource,
com.mchange.v2.resourcepool.BasicResourcePool@411ce9ec is interrupting all Threads waiting on a resource to check out. Will try again in response to new client requests.
2022-08-02 09:56:33,375 [DefaultQuartzScheduler_Worker-1] WARN org.hibernate.engine.jdbc.spi.SqlExceptionHelper - SQL Error: 0, SQLState: null
2022-08-02 09:56:33,375 [DefaultQuartzScheduler_Worker-1] ERROR org.hibernate.engine.jdbc.spi.SqlExceptionHelper - Connections could not be acquired from the underlying database!
2022-08-02 09:56:33,375 [C3P0PooledConnectionPoolManager[identityToken->z8kfluaqp0t6ro20pjuq|3ff5aba]-HelperThread-#0] WARN com.mchange.v2.resourcepool.BasicResourcePool - Having failed to acquire
a resource,
com.mchange.v2.resourcepool.BasicResourcePool@411ce9ec is interrupting all Threads waiting on a resource to check out. Will try again in response to new client requests.
2022-08-02 09:56:33,384 [DefaultQuartzScheduler_Worker-1] ERROR com.redhat.rhn.taskomatic.task.ReportDbUpdateTask - Executing a task threw an exception: org.quartz.JobExecutionException
2022-08-02 09:56:33,385 [DefaultQuartzScheduler_Worker-1] ERROR com.redhat.rhn.taskomatic.task.ReportDbUpdateTask - Message: Unable to update reporting db
2022-08-02 09:56:33,385 [DefaultQuartzScheduler_Worker-1] ERROR com.redhat.rhn.taskomatic.task.ReportDbUpdateTask - Cause: {}
org.hibernate.exception.GenericJDBCException: Unable to acquire JDBC Connection
Caused by: java.sql.SQLException: Connections could not be acquired from the underlying database!
Caused by: com.mchange.v2.resourcepool.CannotAcquireResourceException: A ResourcePool could not acquire a resource from its primary factory or source.
Caused by: org.postgresql.util.PSQLException: The hostname lkv-srvapp39 could not be verified by hostnameverifier PgjdbcHostnameVerifier.
2022-08-02 09:56:33,387 [DefaultQuartzScheduler_Worker-1] ERROR com.redhat.rhn.taskomatic.task.ReportDbUpdateTask - Stack trace:org.quartz.JobExecutionException: Unable to update reporting db [See
nested exception: org.hibernate.exception.GenericJDBCException: Unable to acquire JDBC Connection]
Caused by: org.hibernate.exception.GenericJDBCException: Unable to acquire JDBC Connection
Caused by: java.sql.SQLException: Connections could not be acquired from the underlying database!
Caused by: com.mchange.v2.resourcepool.CannotAcquireResourceException: A ResourcePool could not acquire a resource from its primary factory or source.
Caused by: org.postgresql.util.PSQLException: The hostname lkv-srvapp39 could not be verified by hostnameverifier PgjdbcHostnameVerifier.
2022-08-02 09:57:00,022 [DefaultQuartzScheduler_Worker-2] WARN org.hibernate.orm.deprecation - HHH90000022: Hibernate's legacy org.hibernate.Criteria API is deprecated; use the JPA javax.persistence.criteria.CriteriaQuery
instead
Maybe someone has a better understanding of this stuff and can explain what’s going wrong?
BTW: The messages with “QuartzScheduler” are repeated every minute, independent of the status of “mgr-update-reporting-bunch”.
TIA!
Regards.
Tobias Crefeld.