fusioAuth install is damaged?

richb201

@richb201 said in fusioAuth install is damaged?:

FUSIONAUTH_APP_SILENT-MODE: "false"

I found that I am getting a message from the mysql (this is RDS) that I have reached my limit on errors. Something had changed in my docker-compose.yml. The only way to get to reset the error count was to reboot the mysql server and use my "back in time". I restored numerous days in the past (back to Feb 19th) and now things appear stable again. Seems to be a bug in the fa image? I am using 1.19.7. I will post the current errors up to the github site and see if anyone knows anything. Here they are just for giggles:

2021-02-23 6:23:49.866 AM INFO  com.inversoft.maintenance.MaintenanceModePoller - Poller started to Wait for configuration to be completed.
23-Feb-2021 06:23:49.889 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-9011"]
23-Feb-2021 06:23:49.897 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["https-jsse-nio-9013"]
23-Feb-2021 06:23:49.898 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-127.0.0.1-9019"]
23-Feb-2021 06:23:49.899 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 3295 ms
2021-02-23 6:23:54.907 AM ERROR com.inversoft.maintenance.db.JDBCURL - Could not parse jdbcString [jdbc:mysql:database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com]
2021-02-23 6:23:54.908 AM ERROR com.inversoft.maintenance.db.JDBCMaintenanceModeDatabaseService - Configuration [database.url] is invalid. It must begin with either jdbc:mysql: or jdbc:postgresql:
2021-02-23 6:23:54.927 AM ERROR com.inversoft.maintenance.db.JDBCURL - Could not parse jdbcString [jdbc:mysql:database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com]
2021-02-23 6:23:54.928 AM ERROR com.inversoft.maintenance.db.JDBCMaintenanceModeDatabaseService - Configuration [database.url] is invalid. It must begin with either jdbc:mysql: or jdbc:postgresql:
2021-02-23 6:24:05.346 AM ERROR com.inversoft.maintenance.db.JDBCURL - Could not parse jdbcString [jdbc:mysql:database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com]
2021-02-23 6:24:05.347 AM ERROR com.inversoft.maintenance.db.JDBCMaintenanceModeDatabaseService - Configuration [database.url] is invalid. It must begin with either jdbc:mysql: or jdbc:postgresql:
2021-02-23 6:24:05.359 AM ERROR com.inversoft.maintenance.db.JDBCURL - Could not parse jdbcString [jdbc:mysql:database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com]
2021-02-23 6:24:05.359 AM ERROR com.inversoft.maintenance.db.JDBCMaintenanceModeDatabaseService - Configuration [database.url] is invalid. It must begin with either jdbc:mysql: or jdbc:postgresql:
2021-02-23 6:24:16.748 AM ERROR com.inversoft.maintenance.db.JDBCURL - Could not parse jdbcString [jdbc:mysql:database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com]
2021-02-23 6:24:16.748 AM ERROR com.inversoft.maintenance.db.JDBCMaintenanceModeDatabaseService - Configuration [database.url] is invalid. It must begin with either jdbc:mysql: or jdbc:postgresql:
2021-02-23 6:24:16.762 AM ERROR com.inversoft.maintenance.db.JDBCURL - Could not parse jdbcString [jdbc:mysql:database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com]
2021-02-23 6:24:16.763 AM ERROR com.inversoft.maintenance.db.JDBCMaintenanceModeDatabaseService - Configuration [database.url] is invalid. It must begin with either jdbc:mysql: or jdbc:postgresql:
2021-02-23 6:24:24.892 AM ERROR com.inversoft.maintenance.db.JDBCURL - Could not parse jdbcString [jdbc:mysql:database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com]
2021-02-23 6:24:24.893 AM ERROR com.inversoft.maintenance.db.JDBCMaintenanceModeDatabaseService - Configuration [database.url] is invalid. It must begin with either jdbc:mysql: or jdbc:postgresql:
2021-02-23 6:24:24.901 AM ERROR com.inversoft.maintenance.db.JDBCURL - Could not parse jdbcString [jdbc:mysql:database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com]
2021-02-23 6:24:24.901 AM ERROR com.inversoft.maintenance.db.JDBCMaintenanceModeDatabaseService - Configuration [database.url] is invalid. It must begin with either jdbc:mysql: or jdbc:postgresql:

robotdan

@richb201 said in fusioAuth install is damaged?:

jdbc:mysql://database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com/fusionauthdb

You need to specify the port on the JDBC string. Try this jdbc:mysql://database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com:3306/fusionauthdb.

richb201

Thanks Dan. Interestingly, now when i type localhost:9011, I am getting the attached browser window at http://localhost:9011/maintenance-mode-database-configuration. BTW, I have no idea why it is now almost working. Screenshot from 2021-02-24 06-03-08.png

I am assuming that maint mode is running because fa can;'t connect to my rds mysql.

Into tyhe main mode input screen, now I see a field for port and one for database.

I tried pasting what you gave me,
jdbc:mysql://database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com:3306/fusionauthdb

but now I get

Screenshot from 2021-02-24 06-08-16.png

Then I tried
URL: jdbc:mysql:database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com
Port:3306
databse: fusionauthdb
TLS enabled : no

I still get the error 500.

richb201

@robotdan and Dan I did add the port and the errors in the docker log seems to have gone away. But this causes the following error in the browser. To get around this I need to reboot the rds

A PHP Error was encountered
Severity: Warning

Message: mysqli::real_connect(): (HY000/1129): Host '73.188.125.114' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts'

Filename: mysqli/mysqli_driver.php

Line Number: 203

Backtrace:

File: /app/application/core/MY_Loader.php
Line: 277
Function: database

File: /app/application/controllers/Configure.php
Line: 29
Function: __construct

File: /app/index.php
Line: 325
Function: require_once

A Database Error Occurred
Unable to connect to your database server using the provided settings.

Filename: controllers/Configure.php

Line Number: 29

P.S. I went back to the original connection string and it seems to 1) not get the "is blocked" error above 2) gets the error below in the docker log (this is just a small sampling of it) 3) allows me to connect to http://localhost:9011/maintenance-mode-database-configuration, OK, although I can't figure out what to enter in the maint mode Database Type section.

2021-02-24 11:00:19.765 AM ERROR com.inversoft.maintenance.db.JDBCMaintenanceModeDatabaseService - Configuration [database.url] is invalid. It must begin with either jdbc:mysql: or jdbc:postgresql:
2021-02-24 11:00:19.771 AM ERROR com.inversoft.maintenance.db.JDBCURL - Could not parse jdbcString [jdbc:mysql:database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com]
2021-02-24 11:00:19.771 AM ERROR com.inversoft.maintenance.db.JDBCMaintenanceModeDatabaseService - Configuration [database.url] is invalid. It must begin with either jdbc:mysql: or jdbc:postgresql:
2021-02-24 11:00:29.763 AM ERROR com.inversoft.maintenance.db.JDBCURL - Could not parse jdbcString [jdbc:mysql:database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com]

robotdan

@robotdan said in fusioAuth install is damaged?:

jdbc:mysql://database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com:3306/fusionauthdb

You don't want to paste in the full JDBC string into the UI, it has separate fields for host, database and port.

This string jdbc:mysql://database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com:3306/fusionauthdb could be used in a Docker configuration or directly in the fusionauth.properties file. This JDBC string is constructed as schema:database _type://host:port/database.

If you are using Maintenance Mode, then you will need to use the appropriate parts of this string to fill in the fields.

If you are entering maintenance mode it is because FusionAuth cannot connect to the database as configured, the console messages will describe why it cannot connect, and once you configure the field values in the UI it will tell you if it cannot connect.

richb201

@robotdan said in fusioAuth install is damaged?:

mysql://database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com

I hate to be a pain. Here are the fields from the maint mode.
Screenshot from 2021-02-24 16-18-06.png

and here is the resulting screen:

Screenshot from 2021-02-24 16-19-47.png

did I follow your description of the fields correctly?

maciej.wisniowski

Seems you have a typo in the environment variable name. Instead of DATABASE_ROOT_USER it should be DATABASE_ROOT_USERNAME. Maybe this will resolve the problem.

I have a docker-compose that uses local image of postgresql. For the reference I'm pasting my config below (there are some minor differences in the real version but these should not be relevant):

fusion_db:
    image: postgres:9.6
    environment:
      PGDATA: /var/lib/postgresql/data/pgdata
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: some.password
    networks:
      - fusion_db
    restart: unless-stopped
    ports:
      - 5432:5432
    volumes:
      - fusion_db_data:/var/lib/postgresql/data

  fusionauth:
    image: fusionauth/fusionauth-app:1.24.0
    depends_on:
      - fusion_db
      - elasticsearch
    environment:
      DATABASE_URL: jdbc:postgresql://fusion_db:5432/fusionauth
      DATABASE_ROOT_USERNAME: postgres
      DATABASE_ROOT_PASSWORD: some.password
      DATABASE_USERNAME: fusionauth
      DATABASE_PASSWORD: some.password
      FUSIONAUTH_APP_MEMORY: 256M
      SEARCH_TYPE: elasticsearch
      FUSIONAUTH_APP_RUNTIME_MODE: development
      FUSIONAUTH_APP_SILENT_MODE: "true"
      SEARCH_SERVERS: http://elasticsearch:9200
      FUSIONAUTH_APP_URL: https://auth.mylocal.domain
      FUSIONAUTH_APP_KICKSTART_FILE: docker-fusionauth-kickstart/kickstart.json
    networks:
      - fusion_db
      - fusion_search
      - default
    restart: unless-stopped
    ports:
      - 9011:9011
    volumes:
      - fusion_fa_config:/usr/local/fusionauth/config
      - ./docker-fusionauth-kickstart:/docker-fusionauth-kickstart

volumes:
  fusion_db_data:
  fusion_fa_config:
  (...)

richb201

@maciej-wisniowski Thanks. I took care of that but I still get this error over and over again in the docker log.

2021-02-25 11:59:30.438 AM ERROR com.inversoft.maintenance.db.JDBCMaintenanceModeDatabaseService - Configuration [database.url] is invalid. It must begin with either jdbc:mysql: or jdbc:postgresql:

Here is part of the docker-compose:

environment:
DATABASE_URL: jdbc:mysql://database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com/fusionauthdb
DATABASE_ROOT_USERNAME: ${DATABASE_ROOT_USERNAME} # Enter here the Database username for connection
DATABASE_ROOT_PASSWORD: ${DATABASE_PASSWORD}
# Prior to version 1.19.0, use this deprecated name
# DATABASE_USER: ${DATABASE_USER}
DATABASE_USERNAME: ${DATABASE_USERNAME}
DATABASE_PASSWORD: ${DATABASE_PASSWORD}
# Prior to version 1.19.0, use this deprecated names
# FUSIONAUTH_MEMORY: ${FUSIONAUTH_MEMORY}
# FUSIONAUTH_SEARCH_ENGINE_TYPE: database
# FUSIONAUTH_URL: http://fusionauth:9011
# FUSIONAUTH_RUNTIME_MODE: development
FUSIONAUTH_APP_MEMORY: ${FUSIONAUTH_MEMORY}
FUSIONAUTH_APP_RUNTIME_MODE: development
FUSIONAUTH_APP_URL: http://fusionauth:9011
SEARCH_TYPE: database
FUSIONAUTH_APP_SILENT_MODE: "false"

maciej.wisniowski

Have you tried with original docker image instead of building FA by yourself? Seems (from the first post) that you have some customized dockerfile/image for fusionauth so maybe there is an issue

fusionauth:
  container_name: fusionauth-app
  build : 
    context: ./fusionAuth
    dockerfile: Dockerfile

richb201

@maciej-wisniowski Thanks. I'll ask the guy who rebuilt it why he did that?

One more question. I have a copy of FA running on Ec2 that is able to talk to the RDS mysql fine. How can I see, from the UI, what the connection string being used is?

robotdan

@richb201 said in fusioAuth install is damaged?:

DATABASE_URL: jdbc:mysql://database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com/fusionauthdb

You are still missing the port in this string (as I mentioned above).

DATABASE_URL: jdbc:mysql://database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com:3306/fusionauthdb.

You cannot view the connection string from the UI at runtime.

richb201

@robotdan Dan, putting that port in causes the RDS server to time out. It is repeatable. Is there some other way to see what the connection string is set to on the EC2? Perhaps there is a config file? I do have filezilla attached to that server.

maciej.wisniowski

@richb201 have you tried to connect to this RDS database using any other MySQL client from the same machine? Maybe there is some firewall or permissions issue with RDS

richb201

@maciej-wisniowski When I try this string I get:

ERROR 2005 (HY000): Unknown MySQL server host 'database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com/fusionauthdb' (2)

mysql -h database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com/fusionauthdb -P 3306 -u admin -p

That is why seeing what a working connection string from the EC2 is would solve the problem.

robotdan

ERROR 2005 (HY000): Unknown MySQL server host 'database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com/fusionauthdb'

That is an invalid host name so that error makes sense. The host is database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com and the database is fusionauthdb.

richb201

@robotdan said in fusioAuth install is damaged?:

database-2.cwymdn16cxes.us-east-1.rds.amazonaws.com

What is strange is that I can get into it (RDS mysql) fine from the ec2 directly. I just don't know how that was set up (what connection string is being used)?

As I think I said, using the path and database name you recommend

Screenshot from 2021-02-25 21-25-42.png

Causes a 500 error:
HTTP ERROR 500

maciej.wisniowski

Where is your FA instance? On EC2 or localhost? Seems to be localhost from your screen. If so, then can you connect to RDS from the same machine (localhost probably) as one where is your FA instance? Previously you had errors like: "Host '73.188.125.114' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts'", so maybe it is your localhost IP that is blocked?

Also, if you get error 500 from FA then you should be able to see more detailed logs on the docker-compose console or using docker-compose logs fusionauth

richb201

@maciej-wisniowski thanks for the comment. The fa instance is in EACH container. At least that is my plan. Each container has apache, fa, AWS-CLI, php-fpm, and my application code. The application talks with mysql on RDS just fine. The plan is for fa (in each container) to speak with fusionauthdb which is also up on mysql RDS. So the container on my laptop has its own fa that talks to fusionauthdb on RDS. At least that is the plan!

I will check to see that I have granted permission to my laptop by putting in the question to AWS support for RDS. I am assuming in 2021 that I don't need to serialize access to the fusionauthdb like we use to be required to do in the old mini days?

I have a copy running on my laptop for development purposes. I also plan to have a staging server which is just another copy with xdebug and development logging enabled, and finally the actual deployment server. The servers will all be Lightsail.

richb201

This post is deleted!

richb201

@maciej-wisniowski That was done when we found that there was a bug in

#FROM fusionauth/fusionauth-app:1.19.4

and we replaced it with:

FROM fusionauth/fusionauth-app:1.19.7