kas dump yaml file structure order

[embetrix]

kas-example.yml:

header:
  version: 11
machine: qemux86-64
distro: poky
target:
  - core-image-minimal
repos:
  poky:
    url: "https://git.yoctoproject.org/git/poky"
    branch: scarthgap
    path: "layers"
    layers:
      meta:
      meta-poky:
      meta-yocto-bsp:
  meta-openembedded:
    url: "https://github.com/openembedded/meta-openembedded.git"
    branch: scarthgap
    path: "layers/meta-openembedded"
    layers:
      meta-oe:
      meta-python:
      meta-networking:
      meta-perl:
      meta-python:
      meta-multimedia:
      meta-filesystems:
  meta-swupdate:
    url: "https://github.com/sbabic/meta-swupdate.git"
    branch: scarthgap
    path: "layers/meta-swupdate"
env: 
    DL_DIR         : "${TOPDIR}/downloads"
    SSTATE_DIR     : "${TOPDIR}/sstate-cache"
local_conf_header:
  standard: |
    CONF_VERSION = "2"
    INHERIT += "rm_work"
  debug-tweaks: |
    EXTRA_IMAGE_FEATURES = "debug-tweaks"

kas dump --resolve-refs kas-example.yml > kas-example-fixed.yml

kas-example-fixed.yml:

distro: poky
env:
    DL_DIR: ${TOPDIR}/downloads
    SSTATE_DIR: ${TOPDIR}/sstate-cache
header:
    version: 11
local_conf_header:
    debug-tweaks: |
        EXTRA_IMAGE_FEATURES = "debug-tweaks"
    standard: |
        CONF_VERSION = "2"
        INHERIT += "rm_work"
machine: qemux86-64
repos:
    meta-openembedded:
        branch: scarthgap
        commit: 6de0ab744341ad61b0661aa28d78dc6767ce0786
        layers:
            meta-filesystems:
            meta-multimedia:
            meta-networking:
            meta-oe:
            meta-perl:
            meta-python:
        path: layers/meta-openembedded
        url: https://github.com/openembedded/meta-openembedded.git
    meta-swupdate:
        branch: scarthgap
        commit: ae2f52ce3c8ee742218715d1d542eaae3b4ca19c
        path: layers/meta-swupdate
        url: https://github.com/sbabic/meta-swupdate.git
    poky:
        branch: scarthgap
        commit: 4b07a5316ed4b858863dfdb7cab63859d46d1810
        layers:
            meta:
            meta-poky:
            meta-yocto-bsp:
        path: layers
        url: https://git.yoctoproject.org/git/poky
target:
- core-image-minimal

would be possible to keep the same original structure for kas-example-fixed.yml?:

header:
  version: 11
machine: qemux86-64
distro: poky
target:
  - core-image-minimal
repos:
  poky:
    url: "https://git.yoctoproject.org/git/poky"
    branch: scarthgap
    commit: 4b07a5316ed4b858863dfdb7cab63859d46d1810
    path: "layers"
    layers:
      meta:
      meta-poky:
      meta-yocto-bsp:
  meta-openembedded:
    url: "https://github.com/openembedded/meta-openembedded.git"
    branch: scarthgap
    commit: ae2f52ce3c8ee742218715d1d542eaae3b4ca19c
    path: "layers/meta-openembedded"
    layers:
      meta-oe:
      meta-python:
      meta-networking:
      meta-perl:
      meta-python:
      meta-multimedia:
      meta-filesystems:
  meta-swupdate:
    url: "https://github.com/sbabic/meta-swupdate.git"
    branch: scarthgap
    commit: ae2f52ce3c8ee742218715d1d542eaae3b4ca19c
    path: "layers/meta-swupdate"
env: 
    DL_DIR         : "${TOPDIR}/downloads"
    SSTATE_DIR     : "${TOPDIR}/sstate-cache"
local_conf_header:
  standard: |
    CONF_VERSION = "2"
    INHERIT += "rm_work"
  debug-tweaks: |
    EXTRA_IMAGE_FEATURES = "debug-tweaks"

[fmoessbauer]

Hi @embetrix , despite the changed order, both the input and the combined output should be semantically identical. If not, this is a bug. As kas internally represents the configuration as a python dict, the order is currently not preserved.

Do you have a use-case where preserving the order is required (or desired)? Ideally, you could also bring this topic up on the mailing list ([email protected]), as we have a broader audience there.

[embetrix]

@fmoessbauer I agree that the semantic and usage are identical even the order has changed.
My use case is that I would like that for release branches using a fixed dump of the kas yaml file and track the changes easily in Git,
with a changed order it's hard to comprehend.

[embetrix]

I think that kas dump is sorting the yaml keys alphabetically, I believe it can be turned off.

[fmoessbauer]

I think that kas dump is sorting the yaml keys alphabetically, I believe it can be turned off.

By that I expect to get a rather stable ordering. Without sorting, the keys are likely at nondeterministic positions, which makes thinks even more unstable.

At least for json there is a "canonical" format that is stable (keys are sorted, no whitespacing). I don't know if a similar standard also exists for yaml. This however, is likely not the solution you are looking for.

[embetrix]

maybe as an option parameter for kas dump : do not sort ?

[gowufang]

If the kas-example.yml at the beginning is configured in dictionary order, then the layout of kas-example-fixed.yml will not be so different. lol

[fmoessbauer]

@gowufang I re-checked the code and actually found the inconsistency. When doing yaml.dump(), keys are automatically sorted, while json.dump does not sort the keys. I'll propose a change to only sort the keys if --sort is specified on the command line.