Argument passing strategies: structure of arrays or array of structures?

[ljleb]

For the last 2 days, I have been looking for a way to simplify the 2 argument passing strategies we use at the moment in processing and postprocessing. They are:

• flattened args groups, and
• custom control_net_* properties set on p.

I think the project would benefit from unifying these 2 code paths, as it would result in slightly less code to maintain and a slightly smaller quantity of moving pieces.

I realized recently that you can in fact use p.control_net_* properties to control multiple controlnet units, if you use a list instead of builtin types/ndarray:

sd-webui-controlnet/scripts/controlnet.py

Lines 588 to 591 in fed7a52

	if not isinstance(obj, list):
	return obj if not strict or idx == 0 else None
	elif idx < len(obj):
	return obj[idx]

The external_code api takes a bit of a different approach: instead of using a struct of arrays, it uses an array of structs to represent controlnet layers/processing units.

After pondering on this, it seems to me that #469 may be more the product of my personal arbitrary preferences when it comes to code, than an appropriate contribution. Array of structures (i.e. the design used in scripts/external_code.py) looks to me like better API design in this case, but others may disagree. In particular, with structure of arrays, (i.e. the design currently in use in scripts/controlnet.py to read p.control_net_* properties) the lists not being necessarily all the same length itches for me, even though we may never run into issues using this approach.

So my question is, where do we go from here? So far I see these options:

1. This does not really matter, there are other priorities right now. Keep supporting the 2 argument passing strategies we have currently, namely a) flattened args groups and b) custom properties defined on p.
2. Accept only flattened args groups as input to cn_script.process and .postprocess, and deprecate/guide away from/remove properties defined on p.
3. Same as 2., but instead accept only a list of external_code.ControlNetUnit as input. I found a way to refactor the gradio components in this way. This is what Deprecate /controlnet/*2img web API in favor of new alwayson_scripts support #527 does.

Something else worth noting is that eventually the webui will start supporting always-on scripts in the regular /sdapi/v1/*2img routes. (see the related PR AUTOMATIC1111/stable-diffusion-webui#8187 and the corresponding issue AUTOMATIC1111/stable-diffusion-webui#8253) Depending on how this is implemented in the webui repo, the way the web api is called with controlnet may change radically for users. Under 3., the api could stay practically the same, facilitating route migration. As a1111 is not active currently, this PR could be a distraction though. We don't know when this code will be merged, if it ever will.