Support arrays as values in JSON configuration source

[ChengTian]

Need to figure out a proper representation of arrays when they used as values. Also need to figure out the corresponding things in xml and ini files.

[tillig]

Any update on this? It seems like a minimum requirement for all but the very simplest configuration. I'd like to know if I can count on it being in the release or if I'm going to have to roll my own config system.

[sslava]

For now config values are stored in Dictionary<string, string>. If we could change it to Dictionary<string, object> it could simplify support of custom data type deserialization. At least for .json config files. And use generic T Get<T>() for values retrieval, as @Kieranties mentioned in #130.

[glennc]

@victorhurdugaci Assigning this to you so that getting a design together for it doesn't fall through the cracks.

[sslava]

Hi folks! Any progress on this? :)

[glennc]

Can you guys drop me some examples of what you want to write in the config files to help drive this? Examples of the types of configuration, simple and complex ideally.

[tillig]

My primary use case right now is Autofac configuration. In that, you can configure different components and modules to be registered with Autofac. For example, I might imagine something like this:

{
  "components": [
    {
      "type": "Some.ConcreteType",
      "services": [
        "First.Exposed.IInterface",
        "Second.Exposed.IInterface"
      ],
      "parameters": {
        "simpleParam": "SimpleValue",
        "listParam": [1, 2, 3],
        "dictParam": {
          "name": "value"
        }
      }
    },
    {
      "type": "Some.ConcreteType",
      "services": ["Different.IInterface"]
    }
  ],
  "modules": [
    "First.ModuleType",
    "Second.ModuleType",
    "Third.ModuleType"
  ]
}

The interesting bits in that example from an ordinal list perspective:

• A concrete type can expose 0 or more services.
• You can expose the same concrete type under different interfaces using different configurations. Notice in this case we expose the same type, but the first setup has some parameters that configure the registration; and the second setup doesn't have those parameters and exposes it as a different interface.
• Parameter values for a component registration may be scalar, list, or dictionary values. Via XML we've previously been able to accommodate for all three scenarios.
• Modules (programmatic configuration bundles) are just a list of types that need to be treated like modules. There's no key/value nature to them, it's just a list of type names.

[almightyju]

I've just ran into this problem myself, I'd like to use this config but I can't. I think I'd be happy with at least basic types support for arrays initially (as in strings, numbers)

{
    "Cameras": {
        "Addresses": [ "192.168.0.12", "192.168.0.24" ]
    }    
}

[kevinkuszyk]

For us in R4MVC it is just arrays of basic types too - strings in particular:

{
    "staticFilesFolders": [ "Scripts", "Content" ]
}

[MFFoX]

👍 Looking forward to seeing this get implemented. I have a lot of need for this functionality

[victorhurdugaci]

@davidfowl @lodejard @glennc let's discuss this today in the design meeting and close on the spec. We still need to figure out what to do with the configurations that don't support arrays by default (e.g env variables)

[glennc]

What if this:

 "modules": [
    "First.ModuleType",
    "Second.ModuleType",
    "Third.ModuleType"
  ]

is just syntax sugar for this:

 "modules": {
    "0":"First.ModuleType",
    "1":"Second.ModuleType",
    "2":"Third.ModuleType"
  }

You iterate over the values of the collection by using GetSubKey("modules"). In Environment variables you can override a specific value with with the key Modules:0. So you would need to write out the entire thing in the environment variable keys.

Should also note that the following is consumable with the same code as the one above, assuming we support this syntax as the way to go:

 "modules": {
    "Label for first module type":"First.ModuleType",
    "Label for second module type":"Second.ModuleType",
    "Label for third module type":"Third.ModuleType"
  }

[tillig]

I don't feel strongly about whether it's just a syntactic sugar for auto-generated numeric "keys." That should work. As long as there is no unique constraint enforced on the values it should be just fine.

• How would I know how many there are? While there seems to be a way to get all the subkeys under the current configuration object, there doesn't seem to be a way to get all the keys immediately inside the object. It's like you have to "know" what's in there.
• Would there also be syntactic sugar for element manipulation (add/remove/change)? For Autofac config it's all pretty read-only, but it doesn't make sense that I'd have the ability to get/set other items and not ordinal collections. (And it'd be sort of annoying to have to manually allocate the index into the collection for an ordinal set like that.)

[glennc]

GetSubKeys returns an IEnumerable of KeyValue pairs, which you can count. You might need to know what keys could contain collections, but you would need to know that anyway right? or am I missing something?

Set is interesting, it doesn't actually make any changes to the configuration source. There is no way to commit config changes back to the source. All you are doing is updating the configuration object you call it on. Having Set support a collection though is probably fine.

[tillig]

I could be missing something with GetSubKey("modules") then. Could you maybe post an example? I imagined usage would be like...

var modules = config.GetSubKey("modules");
foreach(var key in modules.Keys)
{
  // get the string value associated with the "fake index" key
  var module = modules[key];
}

But there is no Keys property; instead there's GetSubKeys() so it sounds like it might be something more like...

var modules = config.GetSubKey("modules");
foreach(var kvp in modules.GetSubKeys())
{
  // Is this right?
  var module = modules[kvp.Key];
}

I'm not sure if I'm even close, there. I figure you've got something in mind for that "iterate over the list" thing so maybe if you could post it just for clarification, it'd help solidify things.

[Kieranties]

I've recently written a small POC with 'module' discovery that utilised the following pattern:

Config

{
    "Installers": {
        "Include": {
            "Module1": "My.Custom.Installer.Module1",
            "Module2": "My.Custom.Installer.Module2",
            "Module3": "My.Custom.Installer.Module3"
        }
    }
}

Code

var configRoot = "Installers:Include";

var includes = configuration.GetSubKeys(configRoot)
  .Select(kvp => kvp.Key)
  .Select(k => configuration.Get($"{configRoot}:{k}"));

includes then has the three module type values

Concerns

There's a few things I don't like about this pattern.

• The format the configuration is in supports the concept of an array/list. It feels wrong to be bastardising another aspect of the syntax to do the same thing
• Because we have a key and a value, confusion can arise from which one to use. Is it the key that should be added to the list or the value?
• Overriding values from other configurations: The same key must be used consistently throughout all configs. If a dev simply numbered the entries in the main config and a second dev introduced an override in another config, how can they be certain the key '2' was the correct entry to override?

@glennc I just re-read your comment. Ultimately if the following

"modules": [
    "First.ModuleType",
    "Second.ModuleType",
    "Third.ModuleType"
  ]

could be accessed as

config.Get("modules:0"); //=> "First.ModuleType"
config.Get<IEnumerable<string>>("modules"); //=> "First.ModuleType", "Second.ModuleType", "Third.ModuleType"

That would be awesome!

[tillig]

@Kieranties mentioned something I didn't consider - overrides. If you merge two ordinal collections together... do you get a union of all the elements or do you get just the last collection to get merged in?

[victorhurdugaci]

Below is a summary of the discussion and the spec.

TL;DR;

• Only json supports array. The magic will happen at options level, not config.
• Sorting in integers first, everything after
• Override happens at item level. Same index => overwrite

How it looks like

All the configurations below are equivalent:

json

"ip": [
    "1.2.3.4",
    "3.4.5.6",
    "2.5.643.4"
]

which is actually syntactic sugar for

"ip": {
    "0": "1.2.3.4",
    "1": "3.4.5.6",
    "2": "2.5.43.4"
}

For json arrays, the indices are implied to be 0, 1, 2, ...

environment variables

SET ip:0 = 1.2.3.4
SET ip:1 = 3.4.5.6
SET ip:2 =  2.5.43.4

xml

<ip name="0">1.2.3.4</ip>
<ip name="1">3.4.5.6</ip>
<ip name="2">2.5.43.4</ip>

ini

[addresses]
ip:0 = 1.2.3.4
ip:1 = 3.4.5.6
ip:2 = 2.5.43.4

command line arguments

--ip:0=1.2.3.4 --ip:1=3.4.5.6 --ip:2=2.5.43.4

Usage

Configuration config = new Configuration(<sources>);

var subkeys = config.GetSubKeys("ip");

foreach(var ip in subkeys)
{
    // Might change ToString to be the same as Get(null)
    Console.WriteLine(ip.Value.Get(null));
}

Overwrite and concat rules

Overwrite

The last one wins at index level

If this comes first

"ip": [
    "1.2.3.4",
    "3.4.5.6",
    "2.5.643.4"
]

and this second

<ip name="1">0.0.0.0</ip>

the result is:

1.2.3.4
0.0.0.0
2.5.643.4

Because "3.4.5.6" gets index "1" and gets overwritten by the value from xml (which has name = "1")

Concat/append

If you want to append, use named indices to avoid conflicts

If this comes first

"ip": [
    "1.2.3.4",
    "3.4.5.6",
    "2.5.643.4"
]

and this second

<ip name="john">0.0.0.0</ip>

the result is

1.2.3.4
3.4.5.6
2.5.643.4
0.0.0.0

Sorting

• Integer-only indices come first and are sorted numerically.
• Everything else comes after sorted using the default string sort
• For json arrays, the items are kept in the order in which they appear in the json file

Caveats

"ip": [
    "1.2.3.4",
    "3.4.5.6",
    "2.5.643.4"
]

and

"ip": "0.0.0.0"

are perfectly valid and the config system will read them differently. The second "ip" is not an array of one element.

Options

Options will support model binding to arrays. Will be part of the same change and that's where we encourage everyone to consider arrays. Config will not have arrays as first class citizens.

[lodejard]

The ini example would be:

[ip]
0=1.2.3.4
Etc

Because in the other examples "addresses:" isn't part of the key

Sent from mobile device

---

From: Victor Hurdugacimailto:[email protected]
Sent: ‎4/‎21/‎2015 4:38 PM
To: aspnet/Configurationmailto:[email protected]
Cc: Louis DeJardinmailto:[email protected]
Subject: Re: [Configuration] Support arrays as values in JSON configuration source (#115)

Below is a summary of the discussion and the spec.

TL;DR;

• Only json supports array. The magic will happen at options level, not config.
• Sorting in integers first, everything after
• Override happens at item level. Same index => overwrite

How it looks like

All the configurations below are equivalent:

json

"ip": [
"1.2.3.4",
"3.4.5.6",
"2.5.643.4"
]

which is actually syntactic sugar for

"ip": {
"0": "1.2.3.4",
"1": "3.4.5.6",
"2": "2.5.43.4"
}

For json arrays, the indices are implied to be 0, 1, 2, ...

environment variables

SET ip:0 = 1.2.3.4
SET ip:1 = 3.4.5.6
SET ip:2 = 2.5.43.4

xml

1.2.3.4
3.4.5.6
2.5.43.4

ini

[addresses]
ip:0 = 1.2.3.4
ip:1 = 3.4.5.6
ip:2 = 2.5.43.4

command line arguments

--ip:0 = 1.2.3.4 --ip:1 = 3.4.5.6 --ip:2 = 2.5.43.4

Usage

Configuration config = new Configuration();

var subkeys = config.GetSubKeys("ip");

foreach(var ip in subkeys)
{
// Might change ToString to be the same as Get(null)
Console.WriteLine(ip.Value.Get(null));
}

Overwrite and concat rules

Overwrite

The last one wins at index level

If this comes first

"ip": [
"1.2.3.4",
"3.4.5.6",
"2.5.643.4"
]

and this second

0.0.0.0

the result is:

1.2.3.4
0.0.0.0
2.5.643.4

Because "3.4.5.6" gets index "1" and gets overwritten by the value from xml (which has name = "1")

Concat/append

If you want to append, use named indices to avoid conflicts

If this comes first

"ip": [
"1.2.3.4",
"3.4.5.6",
"2.5.643.4"
]

and this second

0.0.0.0

the result is

1.2.3.4
3.4.5.6
2.5.643.4
0.0.0.0

Sorting

• Integer-only indices come first and are sorted numerically.
• Everything else comes after sorted using the default string sort
• For json arrays, the items are kept in the order in which they appear in the json file

Caveats

"ip": [
"1.2.3.4",
"3.4.5.6",
"2.5.643.4"
]

and

"ip": "0.0.0.0"

are perfectly valid and the config system will read them differently. The second "ip" is not an array of one element.

Options

Options will support model binding to arrays. Will be part of the same change and that's where we encourage everyone to consider arrays. Config will not have arrays as first class citizens.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/115#issuecomment-94975165.

[pholly]

In Asp.net Core 1.1 the syntax to get the IEnumerable is

Configuration.GetSection("Key").GetChildren()

I use this to store allowed origins for CORS requests:

services.AddCors(options =>
  {
     options.AddPolicy("AllowSpecificOrigins", builder =>
      {
          builder.WithOrigins(Configuration.GetSection("Auth:Origins").GetChildren().Select(c => c.Value).ToArray())
           .AllowAnyHeader()
           .AllowAnyMethod();
       });
     });

Would be nice if this were in the documentation.

[mguinness]

@pholly Another approach is to use the Bind method as follows:

var origins = new List<string>();
Configuration.GetSection("Auth:Origins").Bind(origins);