Support separator in the provider

[antoineozenne-at-leocare]

It would be great if we could define the separator to use in the AzureAppConfigurationProvider resource. Currently, the json mounted in the configmap is not the same as the one imported with the az-cli. Here is my use case.

I want to import the following json file in APPCS to mount it in a pod:

{
  "log": {
    "format": "json",
    "level": "debug"
  }
}

For that, I use this command:

az appconfig kv import --label mylabel --separator : --name myappcs --source file --path myfile.json --format json --prefix myprefix/

Then when I use this AzureAppConfigurationProvider:

apiVersion: azconfig.io/v1beta1
kind: AzureAppConfigurationProvider
metadata:
  name: myappcsp
  namespace: myns
spec:
  auth:
    workloadIdentity:
      managedIdentityClientId: XXX
  endpoint: XXX
  keyValues:
    selectors:
    - keyFilter: myprefix/*
      labelFilter: mylabel
    trimKeyPrefixes:
    - myprefix/
  target:
    configMapData:
      key: config.json
      type: json
    configMapName: mycm

The data in the configmap looks like:

{
  "log:format": "json",
  "log:level": "debug",
}

I would set the separator to use in the AzureAppConfigurationProvider resource, but it seems not possible.

[zhenlan]

@antoineozenne-at-leocare, thanks for reporting this.

@linglingye001 @RichardChen820 can you please take a look?

[linglingye001]

Thank you for bringing this issue to our attention. I followed the same steps but was unable to reproduce the issue on my end.

The key values in APPCS looks like this when I use az appconfig kv import command :

{"key": "myprefix/log:format", "value": "json", "label": "mylabel"}
{"key": "myprefix/log:level", "value": "debug", "label": "mylabel"}

And the data in configmap generated by kubernetes provider :

config.json:
----
{"log:format":"json","log:level":"debug"}

Can you please provide more details about the issue you're facing? It will help us better understand the problem and work towards a resolution. Thanks!

[antoineozenne-at-leocare]

Thank you for bringing this issue to our attention. I followed the same steps but was unable to reproduce the issue on my end.

The key values in APPCS looks like this when I use az appconfig kv import command :

{"key": "myprefix/log:format", "value": "json", "label": "mylabel"}
{"key": "myprefix/log:level", "value": "debug", "label": "mylabel"}

And the data in configmap generated by kubernetes provider :

config.json:
----
{"log:format":"json","log:level":"debug"}

Can you please provide more details about the issue you're facing? It will help us better understand the problem and work towards a resolution. Thanks!

@linglingye001 @RichardChen820 @zhenlan , I'm sorry, I made a mistake in the description of my issue. I have edited it. I get the same data as you in my configmap (with the colon as separator, and not a slash).

The problem is not the separator itself, it is the format of the json key (with the colon in the key): {"a:b": "c"} instead of {"a": {"b": "c"}}.

If I export the data with the App Configuration via the Azure Portal, I get {"a": {"b": "c"}}. But in the configmap, I get {"a:b": "c"}.

[RichardChen820]

Do you have a special reason for having to use a hierarchical structure configuration in configMap?

We do support re-construct the configurations into hierarchical structure in exporting scenario since people may take the exported file for other usage, we make a symmetric experience for import/export. But in Kubernetes scenario, it do nothing with export, since the applications do have the ability to understand the flattened configuration, so seems a little bit overkill to re-construct them.

[antoineozenne-at-leocare]

I don't know if this is part of the json spec, but my applications don't understand the flattened configuration. I don't know if we can admit that all applications have the ability to understand the flattened configuration. 🤔

[RichardChen820]

What language/platform is your application built based on?

[antoineozenne-at-leocare]

It is JS and Python. But I can't easily request a code update for that (we don't have the control over all the code).

[RichardChen820]

That sounds reasonable to me. Adding a separator section in the CRD schema and the provider re-constructs the hierarchical json honor with the separator, will this solve your problem?

apiVersion: azconfig.io/v1beta1
kind: AzureAppConfigurationProvider
metadata:
  name: myappcsp
  namespace: myns
spec:
  auth:
    workloadIdentity:
      managedIdentityClientId: XXX
  endpoint: XXX
  keyValues:
    selectors:
    - keyFilter: myprefix/*
      labelFilter: mylabel
    trimKeyPrefixes:
    - myprefix/
  target:
    configMapData:
      key: config.json
      type: json
      hierarchical: 
          enabled: true # default is false
          separator: ":"
    configMapName: mycm

[antoineozenne-at-leocare]

That sounds reasonable to me. Adding a separator section in the CRD schema and the provider re-constructs the hierarchical json honor with the separator, will this solve your problem?

apiVersion: azconfig.io/v1beta1
kind: AzureAppConfigurationProvider
metadata:
  name: myappcsp
  namespace: myns
spec:
  auth:
    workloadIdentity:
      managedIdentityClientId: XXX
  endpoint: XXX
  keyValues:
    selectors:
    - keyFilter: myprefix/*
      labelFilter: mylabel
    trimKeyPrefixes:
    - myprefix/
  target:
    configMapData:
      key: config.json
      type: json
      hierarchical: 
          enabled: true # default is false
          separator: ":"
    configMapName: mycm

Yes, that sounds perfect to me! Thanks!

[landintrees]

@RichardChen820 Are you also able to add this feature to yaml configmap types as well? I.e.:

In AppConfig:

loglevel:global:info
loglevel:service1:debug

would become this in the configmap:

loglevel:
  global: info
  service1: debug

Our use case is that we want to merge helm chart overrides using the appconfig provider and it's not really viable for us to flatten all the values. Thanks!

[RichardChen820]

@landintrees thanks for letting us know your requirement. Yeah, we will support for both yaml and json.

It will be supported in the coming v1.1.0, please stay tuned.

[RichardChen820]

@antoineozenne-at-leocare @landintrees
A new field separator is added under spec.target.configMapData section in v1.1.0 k8s provider for generating hierarchical json/yaml format configMap.

apiVersion: azconfig.io/v1
kind: AzureAppConfigurationProvider
metadata:
  name: myappcsp
  namespace: myns
spec:
  auth:
    workloadIdentity:
      managedIdentityClientId: XXX
  endpoint: XXX
  keyValues:
    selectors:
    - keyFilter: myprefix/*
      labelFilter: mylabel
    trimKeyPrefixes:
    - myprefix/
  target:
    configMapData:
      key: config.json
      type: json
      separator: ':'
    configMapName: mycm