"Unrecognized configuration key" regression in 1.11.0.CR1

[vsevel]

Describe the bug
When running in native mode with 1.11.0.CR1, we get "Unrecognized configuration key" warnings on a list of properties.
those properties are all "quarkus" properties (we do not have any warnings on our applicative properties).
some of those properties come from src/main/resources/application.yml, some properties come from runtime provided "application.yml", and some properties are not set at all in any of these 2 files.
We do not have this behavior in 1.10.5 native, or 1.11.0.CR1 jvm.
The interesting thing is that the program is running normally. that is those properties are correctly read from the config.

Expected behavior
No warnings "Unrecognized configuration key" in native mode.

Actual behavior
here is what we get at startup:

__  ____  __  _____   ___  __ ____  ______ 
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/ 
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.logserver.display-env-variables" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.freemarker.directive.base64" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.log.console.darken" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.freemarker.directive.mongodb_uri" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.freemarker.default-encoding" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.freemarker.template-exception-handler" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.freemarker.log-template-exceptions" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.smbj.hostname" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.freemarker.boolean-format" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.freemarker.object-wrapper-expose-fields" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.log.filter."org.jboss.threads".if-starts-with" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.freemarker.directive.json" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.application.name" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.freemarker.number-format" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.log.category."io.quarkus.vault".level" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.freemarker.wrap-unchecked-exceptions" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.log.category."com.lodh.arte.ocpdeploy".level" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.logserver.erl-url" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.freemarker.fallback-on-null-loop-variable" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.log.filter."org.hibernate.validator.internal.util.Version".if-starts-with" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.profile" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.log.min-level" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.freemarker.file-paths" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.freemarker.directive.indent" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.application.version" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:14,762 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.freemarker.directive.yaml" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo
2021-01-11 10:25:15,203 INFO  [com.lod.art.log.LogServerHandlerRecorder] (main) Initializing LogServer Agent
2021-01-11 10:25:15,215 INFO  [io.quarkus] (main) ocpdeploy 4.1.0-SNAPSHOT native (powered by Quarkus 1.11.0.CR1) started in 0.463s. 
2021-01-11 10:25:15,215 INFO  [io.quarkus] (main) Profile prod activated. 
2021-01-11 10:25:15,215 INFO  [io.quarkus] (main) Installed features: [cdi, config-yaml, freemarker, hibernate-validator, kubernetes-client, logserver, mutiny, openshift-client, rest-client, rest-client-jackson, smallrye-context-propagation, smbj, vault]

From all of these properties we have 3 cases:

properties coming from src/main/resources/application.yaml

quarkus:
  log:
    min-level: DEBUG
  smbj:
    hostname: ${ocpdeploy.share.hostname:""}
  freemarker:
    file-paths: ${ocpdeploy.workspace-path}
    default-encoding: UTF-8
    template-exception-handler: rethrow
    log-template-exceptions: false
    wrap-unchecked-exceptions: true
    fallback-on-null-loop-variable: true
    boolean-format: c
    number-format: computer
    object-wrapper-expose-fields: true
    directive:
      base64: ...
      indent: ...
      yaml: ...
      json: ...
      mongodb_uri: ...
  logserver:
    erl-url: ${ocpdeploy.erl.url}

which is all quarkus properties, except:

  vault:
    kv-secret-engine-version: 1

properties coming from runtime provided application.yaml

                logserver:
                  display-env-variables: false
                log:
                  category:
                    "...":
                      level: DEBUG
                    "io.quarkus.vault":
                      level: DEBUG

which is all quarkus properties except vault properties:

              quarkus:
                vault:
                  url: ...
                  authentication:
                    kubernetes:
                      role: ...
                  secret-config-kv-path:
                    creds: ...

properties not coming from our config either runtime or build time

quarkus.log.console.darken
quarkus.log.filter."org.jboss.threads".if-starts-with
quarkus.application.name
quarkus.log.filter."org.hibernate.validator.internal.util.Version".if-starts-with
quarkus.profile
quarkus.application.version

To Reproduce
We tried to reproduce in a simple "getting started" example, but did not succeed (instead we stumbled upon #14229).
so unfortunately we do not have a reproducer.

I tried printing a few variables at startup (some coming from src/main/resources/application.yml, and some coming from the runtime provided application.yml):

    @ConfigProperty(name = "quarkus.smbj.hostname")
    String quarkusSmbjHostname;

    @ConfigProperty(name = "quarkus.freemarker.default-encoding")
    String quarkusFreemarkerDefaultEncoding;

    @ConfigProperty(name = "quarkus.logserver.display-env-variables")
    Boolean quarkusLogServerDisplayEnvVariables;

    public void execute() throws InterruptedException, TemplateException, NoSuchAlgorithmException, IOException {

        log.info("quarkus.smbj.hostname => " + quarkusSmbjHostname);
        log.info("quarkus.freemarker.default-encoding => " + quarkusFreemarkerDefaultEncoding);
        log.info("quarkus.logserver.display-env-variables => " + quarkusLogServerDisplayEnvVariables);

in jvm mode I get (which is expected):

2021-01-11 18:32:59,313 INFO (main) quarkus.smbj.hostname => ...some value...
2021-01-11 18:32:59,314 INFO (main) quarkus.freemarker.default-encoding => UTF-8
2021-01-11 18:32:59,315 INFO (main) quarkus.logserver.display-env-variables => false

in native mode I get the same values, but with the unexpected warnings, such as:

2021-01-11 17:53:50,404 WARN  [io.qua.config] (main) Unrecognized configuration key "quarkus.smbj.hostname" was provided; it will be ignored; verify that the dependency extension for this configuration is set or that you did not make a typo

so properties are read appropriately (which explains why the program is running normally). the only issue seems to be those invalid warnings.

[ghost]

/cc @geoand

[geoand]

@gsmet do you know anything about this one?

[matthyx]

+1

[gsmet]

Hum. This looks like a blocker. I have no idea where the issue can come from.

@vsevel could you try reverting #13376 and see how it goes?

[gsmet]

@vsevel @matthyx btw, Final release is planned for tomorrow morning so this is a bit time sensitive.

Building master with the PR reverted should be relatively easy by following: https://github.com/quarkusio/quarkus/blob/master/CONTRIBUTING.md#checking-an-issue-is-fixed-in-master .

[gsmet]

/cc @radcortez as it might also be a SmallRye Config regression.

[gsmet]

@vsevel could you also try without the runtime provided config file? Trying to narrow things down.

[vsevel]

@gsmet we are going to try. there might be some challenges in rebuilding master in our corporate env.

[gsmet]

@vsevel also a couple of additional questions:

• which GraalVM version are you using?
• can you try with an application.properties and removing the Config YAML extension?

[vsevel]

Running Quarkus native-image plugin on GraalVM Version 20.3.0 (Java Version 11.0.9+10-jvmci-20.3-b06)

[gsmet]

OK, so if building master is complex for you, maybe try with the 2 experiments:

• remove runtime config file
• move to properties and drop the Config Yaml extension

and let's discuss things from there.

[vsevel]

the quarkus build fails on gradle on our env. there is no knowledge on this. so it is probably not going to work out. if you can provide a debug version of quarkus we can use, that will be easier for us.

I tried moving all "quarkus" properties to the build time file (I left our applicative properties). and that did not resolve the issue.
I will try to move to a properties file now.

[gsmet]

@vsevel also can you iterate through the 1.11 Betas and see which one introduces the issue?

[vsevel]

I removed the runtime config entirely, and I still see the warnings. I will go through the betas. good idea.

[radcortez]

Having a look as well.

[vsevel]

I have the issue with Beta2. trying with beta1.

[geoand]

@vsevel Stuart's fix in #14247 should take care of the issue.

I tried it with another example and the problem went away.

[vsevel]

I have the issue with beta1. is it related to having a bootstrap config? if so #13498 was part of beta1. and that might explain why the vault properties were not subject to the warnings?
this is good news. if you have a published build, I could give it a try.

[geoand]

Yes, it was an issue with bootstrap config

[vsevel]

@stuartwdouglas well done!