Cannot login to FusionAuth dashboard
-
Cannot login to FusionAuth dashboard
Description
Our FusionAuth instance was running with no errors until one day, when logging in to FusionAuth we are redirected to "Return to Login" page everytime.
Observed versions
- Version: 1.45.2
Expected behavior
Being able to log in to FusionAuth and get redirected to the Admin dashboard
Screenshots
Platform
- Device: Desktop
- OS: Windows, Apple
- Browser + version: Chrome Version 134.0.6998.37 (Official Build) (64-bit)
- Database PostgresSQL; Version: 9.6.18-apline
- FusionAuth server & FusionAuth DB are running withing a Kubernetes cluster
Attempted fixes:
- Restart FusionAuth
- Restart Database
- Tried multiple accounts
Additional context
- This problem seemed to occur due to an unexpected restart of the PostgresSQL database.
- The database is up & running.
- Debug log when the database restarted
2025-03-12 03:15:00.232 CET ### Error querying database. Cause: java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 2001ms. 2025-03-12 03:15:00.232 CET ### The error may exist in io/fusionauth/api/domain/InstanceMapper.java (best guess) 2025-03-12 03:15:00.232 CET ### The error may involve io.fusionauth.api.domain.InstanceMapper.retrieve 2025-03-12 03:15:00.232 CET ### The error occurred while executing a query 2025-03-12 03:15:00.232 CET ### Cause: java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 2001ms. at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:30) at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:153) at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:145) at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:140) at org.apache.ibatis.session.defaults.DefaultSqlSession.selectOne(DefaultSqlSession.java:76) at jdk.internal.reflect.GeneratedMethodAccessor6.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.apache.ibatis.session.SqlSessionManager$SqlSessionInterceptor.invoke(SqlSessionManager.java:357) at jdk.proxy2/jdk.proxy2.$Proxy52.selectOne(Unknown Source) at org.apache.ibatis.session.SqlSessionManager.selectOne(SqlSessionManager.java:166) at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:87) at org.apache.ibatis.binding.MapperProxy$PlainMethodInvoker.invoke(MapperProxy.java:145) at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:86) at jdk.proxy2/jdk.proxy2.$Proxy59.retrieve(Unknown Source) at io.fusionauth.api.license.FusionAuthLicenseMetaDataManager.localLookup(FusionAuthLicenseMetaDataManager.java:50) at com.inversoft.license.v2.BaseConfigurationLicenseMetaDataManager.lookup(BaseConfigurationLicenseMetaDataManager.java:30) at com.inversoft.license.v2.local.DefaultLocalLicenseSystem.load(DefaultLocalLicenseSystem.java:35) at com.inversoft.license.v2.DefaultLicenseProvider$LocalReloader.run(DefaultLicenseProvider.java:131) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 2001ms. at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:696) at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:181) at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:146) at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:100) at org.apache.ibatis.transaction.jdbc.JdbcTransaction.openConnection(JdbcTransaction.java:139) at org.apache.ibatis.transaction.jdbc.JdbcTransaction.getConnection(JdbcTransaction.java:61) at org.apache.ibatis.executor.BaseExecutor.getConnection(BaseExecutor.java:337) at org.apache.ibatis.executor.SimpleExecutor.prepareStatement(SimpleExecutor.java:86) at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:62) at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:325) at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:156) at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:109) at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:89) at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:151) ... 23 common frames omitted Caused by: org.postgresql.util.PSQLException: This connection has been closed. at org.postgresql.jdbc.PgConnection.checkClosed(PgConnection.java:907) at org.postgresql.jdbc.PgConnection.setNetworkTimeout(PgConnection.java:1649) at com.zaxxer.hikari.pool.PoolBase.setNetworkTimeout(PoolBase.java:561) at com.zaxxer.hikari.pool.PoolBase.isConnectionDead(PoolBase.java:168) at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:170) ... 35 common frames omitted 2025-03-12 03:15:01.330 CET 2025-03-12 02:15:01.329 AM ERROR io.fusionauth.api.service.system.DefaultAsyncTaskManager - An exception occurred while managing an async task. 2025-03-12 03:15:01.330 CET org.apache.ibatis.exceptions.PersistenceException: 2025-03-12 03:15:01.330 CET ### Error querying database. Cause: java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 2000ms. 2025-03-12 03:15:01.330 CET ### The error may exist in io/fusionauth/api/domain/LockMapper.java (best guess) 2025-03-12 03:15:01.330 CET ### The error may involve io.fusionauth.api.domain.LockMapper.lock 2025-03-12 03:15:01.330 CET ### The error occurred while executing a query- Debug log after restart
2025-03-12T09:14:09.786694464Z {} 2025-03-12T09:14:09.786718924Z 2025-03-12 09:14:09.786 AM WARN com.inversoft.configuration.BasePropertiesFileInversoftConfiguration - Your FusionAuth configuration file [/usr/local/fusionauth/config/fusionauth.properties] needs attention. Here are the warnings: 2025-03-12T09:14:09.786772574Z {} 2025-03-12T09:14:09.786782694Z - You are using a deprecated configuration property name of [FUSIONAUTH_APP_HTTP_PORT]. The new allowed names for that property are [fusionauth-app.http.port] 2025-03-12T09:14:09.786789094Z - You are using a deprecated configuration property name of [FUSIONAUTH_APP_HTTP_PORT]. The new allowed names for that property are [fusionauth-app.http.port] 2025-03-12T09:14:09.786794554Z {} 2025-03-12T09:14:09.788385584Z 2025-03-12 09:14:09.788 AM INFO com.inversoft.jdbc.hikari.DataSourceProvider - Connecting to PostgreSQL database at [jdbc:postgresql://divizend-auth-db-0:5432/fusionauth] 2025-03-12T09:14:09.791886714Z 2025-03-12 09:14:09.791 AM WARN com.zaxxer.hikari.HikariConfig - HikariPool-1 - idleTimeout has been set but has no effect because the pool is operating as a fixed size pool. 2025-03-12T09:14:09.794193314Z 2025-03-12 09:14:09.793 AM INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Starting... 2025-03-12T09:14:09.819821924Z 2025-03-12 09:14:09.819 AM INFO com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@6cbe68e9 2025-03-12T09:14:09.831727314Z 2025-03-12 09:14:09.831 AM INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Start completed. 2025-03-12T09:14:11.268084494Z 2025-03-12 09:14:11.267 AM INFO com.inversoft.scheduler.DefaultScheduler - Starting up scheduler 2025-03-12T09:14:11.268828044Z 2025-03-12 09:14:11.268 AM INFO com.inversoft.scheduler.DefaultScheduler - Scheduler is running 2025-03-12T09:14:11.439808283Z 2025-03-12 09:14:11.439 AM INFO io.fusionauth.api.service.system.NodeService - Node [deb1186a-32c4-493e-83a7-a5299b66f6a8] added with address [http://10.52.0.28:80] 2025-03-12T09:14:12.945987592Z 2025-03-12 09:14:12.944 AM ERROR com.inversoft.scheduler.LogAndRetainFailureHandler - The scheduled service [class io.fusionauth.api.service.cache.KeyCacheLoader] failed but will be re-run. 2025-03-12T09:14:12.946030462Z io.fusionauth.pem.PEMDecoderException: java.security.InvalidKeyException: Could not decode the X.509 public key. Expected at 2 values in the DER encoded sequence but found [2] at io.fusionauth.pem.PEMDecoder.decode(PEMDecoder.java:126) at io.fusionauth.pem.domain.PEM.decode(PEM.java:111) at io.fusionauth.api.service.cache.KeyCacheLoader.internalLoad(KeyCacheLoader.java:101) at com.inversoft.cache.BaseCacheLoader.load(BaseCacheLoader.java:21) at io.fusionauth.api.service.cache.KeyCacheLoader.run(KeyCacheLoader.java:58) at com.inversoft.scheduler.InjectableRunnable.run(InjectableRunnable.java:56) at com.inversoft.scheduler.DefaultScheduler.<init>(DefaultScheduler.java:70) at com.inversoft.scheduler.DefaultScheduler$$FastClassByGuice$$208640461.GUICE$TRAMPOLINE(<generated>) at com.inversoft.scheduler.DefaultScheduler$$FastClassByGuice$$208640461.apply(<generated>) at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:82) at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114) at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91) at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:300) at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:169) at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:45) at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:60) at com.google.inject.internal.InternalInjectorCreator.loadEagerSingletons(InternalInjectorCreator.java:213) at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:186) at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:113) at com.google.inject.Guice.createInjector(Guice.java:87) at com.google.inject.Guice.createInjector(Guice.java:69) at com.google.inject.Guice.createInjector(Guice.java:59) at org.primeframework.mvc.guice.GuiceBootstrap.initialize(GuiceBootstrap.java:60) at org.primeframework.mvc.BasePrimeMain.hup(BasePrimeMain.java:69) at org.primeframework.mvc.BasePrimeMain.start(BasePrimeMain.java:100) at io.fusionauth.app.FusionAuthMain.main(FusionAuthMain.java:27) Caused by: java.security.InvalidKeyException: Could not decode the X.509 public key. Expected at 2 values in the DER encoded sequence but found [2] at io.fusionauth.pem.PEMDecoder.decode_X_509(PEMDecoder.java:334) at io.fusionauth.pem.PEMDecoder.decode(PEMDecoder.java:112) ... 26 common frames omitted 2025-03-12T09:14:15.700435663Z 2025-03-12 09:14:15.699 AM INFO io.fusionauth.api.service.system.NodeService - Node [deb1186a-32c4-493e-83a7-a5299b66f6a8] promoted to master at [2025-03-12T09:14:15.699888323Z], the previous master Node [cf39ac34-f49e-4394-8721-8a2f8bfad22c] has been shutdown or removed 2025-03-12T09:14:16.467362998Z 2025-03-12 09:14:16.466 AM INFO io.fusionauth.app.primeframework.FusionHTTPContextAuthSetup - Initializing the FusionAuth HTTP Context. 2025-03-12T09:14:16.674891576Z 2025-03-12 09:14:16.674 AM INFO org.primeframework.mvc.PrimeMVCRequestHandler - Initializing Prime 2025-03-12T09:14:16.676199576Z 2025-03-12 09:14:16.675 AM INFO org.primeframework.mvc.PrimeMVCRequestHandler - Initializing Prime 2025-03-12T09:14:16.676563276Z 2025-03-12 09:14:16.676 AM INFO io.fusionauth.http.server.HTTPServer - Starting the HTTP server. Buckle up! 2025-03-12T09:14:16.684964236Z 2025-03-12 09:14:16.684 AM INFO io.fusionauth.http.server.HTTPServer - HTTP server listening on port [80] 2025-03-12T09:14:16.684994656Z 2025-03-12 09:14:16.684 AM INFO io.fusionauth.http.server.HTTPServer - HTTP server started successfully 2025-03-12T09:14:16.685016076Z 2025-03-12 09:14:16.684 AM INFO io.fusionauth.http.server.HTTPServer - Starting the HTTP server. Buckle up! 2025-03-12T09:14:16.685700126Z 2025-03-12 09:14:16.685 AM INFO io.fusionauth.http.server.HTTPServer - HTTP server listening on port [9012] 2025-03-12T09:14:16.685722766Z 2025-03-12 09:14:16.685 AM INFO io.fusionauth.http.server.HTTPServer - HTTP server started successfully -
@mohamed-aziz-khayati does this occur simply when you visit the page or when you try to login (submit your username and password)?
