Unable to set datasource username and password just via environment variables

[mzuber]

Describe the bug

Up until Quarkus 3.2.4.Final it was possible to set the quarkus.datasource.username and quarkus.datasource.password configuration properties by just providing the corresponding environment variables QUARKUS_DATASOURCE_USERNAME and QUARKUS_DATASOURCE_PASSWORD.

With Quarkus 3.3.0 this configuration setup results in the following error message when using a PostgreSQL database:

java.lang.RuntimeException: Failed to start quarkus
    at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
    at io.quarkus.runtime.Application.start(Application.java:101)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at io.quarkus.runner.bootstrap.StartupActionImpl.run(StartupActionImpl.java:273)
    at io.quarkus.test.junit.QuarkusTestExtension.doJavaStart(QuarkusTestExtension.java:251)
    at io.quarkus.test.junit.QuarkusTestExtension.ensureStarted(QuarkusTestExtension.java:606)
    at io.quarkus.test.junit.QuarkusTestExtension.beforeAll(QuarkusTestExtension.java:655)
    at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.lambda$invokeBeforeAllCallbacks$12(ClassBasedTestDescriptor.java:395)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.invokeBeforeAllCallbacks(ClassBasedTestDescriptor.java:395)
    at org.junit.jupiter.engine.descriptor.ClassBasedTestDescriptor.before(ClassBasedTestDescriptor.java:211)
    ... 53 more

    Caused by:
    java.lang.IllegalStateException: Error starting Liquibase
        at io.quarkus.liquibase.runtime.LiquibaseRecorder.doStartActions(LiquibaseRecorder.java:88)
        at io.quarkus.deployment.steps.LiquibaseProcessor$startLiquibase1744275855.deploy_0(Unknown Source)
        at io.quarkus.deployment.steps.LiquibaseProcessor$startLiquibase1744275855.deploy(Unknown Source)
        ... 67 more

        Caused by:
        java.lang.IllegalStateException: org.postgresql.util.PSQLException: The server requested SCRAM-based authentication, but no password was provided.
            at io.quarkus.liquibase.LiquibaseFactory.createLiquibase(LiquibaseFactory.java:60)
            at io.quarkus.liquibase.LiquibaseFactory_d5de3c3b0b2c3d9a10cad0f6a7c9064403f9e326_Synthetic_ClientProxy.createLiquibase(Unknown Source)
            at io.quarkus.liquibase.runtime.LiquibaseRecorder.doStartActions(LiquibaseRecorder.java:65)
            ... 69 more

            Caused by:
            org.postgresql.util.PSQLException: The server requested SCRAM-based authentication, but no password was provided.

Now the environment variables QUARKUS_DATASOURCE_USERNAME and QUARKUS_DATASOURCE_PASSWORD only work when corresponding configuration properties are present in the application.properties file, too, e.g.

quarkus.datasource.username = dummy
quarkus.datasource.password = dummy

Should this be considered a bug? Or is this the desired behavior and the previous setup with just defining these configuration properties via environment variables should have never worked in the first place?

Expected behavior

Only providing the environment variables QUARKUS_DATASOURCE_USERNAME and QUARKUS_DATASOURCE_PASSWORD is enough to configure username and password for the used datasource.

Actual behavior

The configuration properties quarkus.datasource.username and quarkus.datasource.password need to be present in the application.properties file for the environment variables to have an effect.

How to Reproduce?

In a Quarkus application using for example the quarkus-jdbc-postgresql extension, start a local PostgreSQL database using Docker, configure the data source just via the corresponding environment variables, and start the application:

docker run --name postgres -p 5432:5432  -e "POSTGRES_DB=postgres" -e "POSTGRES_USER=postgres" -e "POSTGRES_PASSWORD=postgres" -d postgres:14-alpine
export _DEV_QUARKUS_DATASOURCE_USERNAME="postgres"
export _DEV_QUARKUS_DATASOURCE_PASSWORD="postgres"
export _DEV_QUARKUS_DATASOURCE_JDBC_URL="jdbc:postgresql://localhost:5432/postgres"
./gradlew quarkusDev

Output of uname -a or ver

No response

Output of java -version

openjdk version "17.0.7" 2023-04-18

GraalVM version (if different from Java)

OpenJDK Runtime Environment GraalVM CE 17.0.7+7.1 (build 17.0.7+7-jvmci-23.0-b12)

Quarkus version or git rev

3.3.0

Build tool (ie. output of mvnw --version or gradlew --version)

Gradle 8.3

Additional information

No response

[mzuber]

Might be related to the changes in #35262. If this is not considered a bug but an expected change for Quarkus 3.3.0 it might be worth mentioning in the Migration-Guide.

[gsmet]

/cc @radcortez

[radcortez]

@mzuber can you please provide a reproducer? I was not able to reproduce it with the provided steps.

I did try with a personal project with the following result:

(no username and password set)

java -jar target/quarkus-app/quarkus-run.jar
__  ____  __  _____   ___  __ ____  ______
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
2023-08-28 17:27:46 WARN  [io.agr.pool]] (agroal-11) Datasource '<default>': The server requested password-based authentication, but no password was provided by plugin null
2023-08-28 17:27:46 INFO  [org.fly.cor.int.lic.VersionPrinter]] (main) Flyway Community Edition 9.21.0 by Redgate
2023-08-28 17:27:46 INFO  [org.fly.cor.int.lic.VersionPrinter]] (main) See release notes here: https://rd.gt/416ObMi
2023-08-28 17:27:46 INFO  [org.fly.cor.int.lic.VersionPrinter]] (main)
2023-08-28 17:27:46 WARN  [io.agr.pool]] (agroal-11) Datasource '<default>': The server requested password-based authentication, but no password was provided by plugin null
2023-08-28 17:27:46 ERROR [io.qua.run.Application]] (main) Failed to start application (with profile [prod]): java.lang.RuntimeException: Failed to start quarkus
	at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
	at io.quarkus.runtime.Application.start(Application.java:101)
	at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:111)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:71)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:44)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:124)
	at io.quarkus.runner.GeneratedMain.main(Unknown Source)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at io.quarkus.bootstrap.runner.QuarkusEntryPoint.doRun(QuarkusEntryPoint.java:61)
	at io.quarkus.bootstrap.runner.QuarkusEntryPoint.main(QuarkusEntryPoint.java:32)
Caused by: org.flywaydb.core.internal.exception.FlywaySqlException: Unable to obtain connection from database: The server requested password-based authentication, but no password was provided by plugin null
------------------------------------------------------------------------------------------------------------------------------------------
SQL State  : 08004
Error Code : 0
Message    : The server requested password-based authentication, but no password was provided by plugin null

	at org.flywaydb.core.internal.jdbc.JdbcUtils.openConnection(JdbcUtils.java:60)
	at org.flywaydb.core.internal.jdbc.JdbcConnectionFactory.<init>(JdbcConnectionFactory.java:74)
	at org.flywaydb.core.FlywayExecutor.execute(FlywayExecutor.java:140)
	at org.flywaydb.core.Flyway.migrate(Flyway.java:140)
	at io.quarkus.flyway.runtime.FlywayRecorder.doStartActions(FlywayRecorder.java:108)
	at io.quarkus.deployment.steps.FlywayProcessor$startActions2035800939.deploy_0(Unknown Source)
	at io.quarkus.deployment.steps.FlywayProcessor$startActions2035800939.deploy(Unknown Source)
	... 13 more
Caused by: org.postgresql.util.PSQLException: The server requested password-based authentication, but no password was provided by plugin null
	at org.postgresql.core.v3.AuthenticationPluginManager.lambda$withEncodedPassword$0(AuthenticationPluginManager.java:110)
	at org.postgresql.core.v3.AuthenticationPluginManager.withPassword(AuthenticationPluginManager.java:81)
	at org.postgresql.core.v3.AuthenticationPluginManager.withEncodedPassword(AuthenticationPluginManager.java:107)
	at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:711)
	at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:203)
	at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:258)
	at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:54)
	at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:263)
	at org.postgresql.Driver.makeConnection(Driver.java:443)
	at org.postgresql.Driver.connect(Driver.java:297)
	at io.agroal.pool.ConnectionFactory.createConnection(ConnectionFactory.java:226)
	at io.agroal.pool.ConnectionPool$CreateConnectionTask.call(ConnectionPool.java:536)
	at io.agroal.pool.ConnectionPool$CreateConnectionTask.call(ConnectionPool.java:517)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at io.agroal.pool.util.PriorityScheduledExecutor.beforeExecute(PriorityScheduledExecutor.java:75)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1126)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

with username and password in env:

export QUARKUS_DATASOURCE_PASSWORD=password
export QUARKUS_DATASOURCE_USERNAME=database
java -jar target/quarkus-app/quarkus-run.jar
__  ____  __  _____   ___  __ ____  ______
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/
2023-08-28 17:29:48 INFO  [org.fly.cor.int.lic.VersionPrinter]] (main) Flyway Community Edition 9.21.0 by Redgate
2023-08-28 17:29:48 INFO  [org.fly.cor.int.lic.VersionPrinter]] (main) See release notes here: https://rd.gt/416ObMi
2023-08-28 17:29:48 INFO  [org.fly.cor.int.lic.VersionPrinter]] (main)
2023-08-28 17:29:48 INFO  [org.fly.cor.int.dat.bas.BaseDatabaseType]] (main) Database: jdbc:postgresql://localhost:5432/database (PostgreSQL 11.5)
2023-08-28 17:29:48 INFO  [org.fly.cor.int.com.DbValidate]] (main) Successfully validated 1 migration (execution time 00:00.022s)
2023-08-28 17:29:48 INFO  [org.fly.cor.int.com.DbMigrate]] (main) Current version of schema "public": 1.0.0
2023-08-28 17:29:48 INFO  [org.fly.cor.int.com.DbMigrate]] (main) Schema "public" is up to date. No migration necessary.

[radcortez]

I've also added some code to make sure that I didn't have the property dotted format defined somewhere:

@Path("/config")
public class ConfigResource {
    @Inject
    SmallRyeConfig config;

    @GET
    public Response get() {
        for (ConfigSource configSource : config.getConfigSources()) {
            String username = configSource.getValue("quarkus.datasource.username");
            String password = configSource.getValue("quarkus.datasource.password");
            System.out.println(configSource.getName() + " "  + username + " " + password) ;
        }
        return Response.noContent().build();
    }
}

BuildTime RunTime Fixed null null
SysPropConfigSource null null
EnvConfigSource database password
PropertiesConfigSource[source=jar:file:///Users/radcortez/Code/personal/quarkus-playground/services/book-api/target/quarkus-app/app/book-api.jar!/application.properties] null null
PropertiesConfigSource[source=jar:file:///Users/radcortez/Code/personal/quarkus-playground/services/book-api/target/quarkus-app/lib/main/io.quarkiverse.microprofile.quarkus-microprofile-3.0.0.Final.jar!/META-INF/microprofile-config.properties] null null
null:null:ServletConfigSource null null
null:null:FilterConfigSource null null
null:ServletContextConfigSource null null
RunTime Defaults null null
default values null null
Quarkus HTTP Host Default Value null null
DefaultValuesConfigSource null null

The values are only found in the EnvConfigSource.

[mzuber]

@radcortez I was also able to reproduce the issue with the liquibase-quickstart:

• Add quarkus-jdbc-postgresql to pom.xml:

  <dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-jdbc-postgresql</artifactId>
  </dependency>

• Adapt the application.properties to not contain any data source username or password configuration:

  # configure your datasource
  quarkus.datasource.db-kind=postgresql
  
  # Liquibase minimal config properties
  quarkus.liquibase.migrate-at-start=true

• Run local PostgreSQL database:

  docker run --name postgres -p 5432:5432  -e "POSTGRES_DB=postgres" -e "POSTGRES_USER=postgres" -e "POSTGRES_PASSWORD=postgres" -d postgres:14-alpine

• Set environment variables to configure datasource:

  export _DEV_QUARKUS_DATASOURCE_USERNAME="postgres"
  export _DEV_QUARKUS_DATASOURCE_PASSWORD="postgres"
  export _DEV_QUARKUS_DATASOURCE_JDBC_URL="jdbc:postgresql://localhost:5432/insights-central"

• Run Quarkus in development mode:

  ./mvnw quarkus:dev

[mzuber]

Setting the environment variables without a profile works, though, i.e.,

export QUARKUS_DATASOURCE_USERNAME=postgres
export QUARKUS_DATASOURCE_PASSWORD=postgres
export QUARKUS_DATASOURCE_JDBC_URL="jdbc:postgresql://localhost:5432/postgres"

will not produce the error and everything works as expected.

[radcortez]

Got it! Thank you.

This will be fixed by smallrye/smallrye-config#987. In the meanwhile has a workaround, please add the corresponding dotted property names in application.properties. The properties can even be set as empty, since the goal is to override them.