From cb3e26ce4449b005fa8a0f46f230bc70ff3ebaf6 Mon Sep 17 00:00:00 2001 From: Zach White Date: Sun, 10 Jan 2021 09:33:05 -0800 Subject: [PATCH 1/2] describe how data driven configuration works --- docs/_summary.md | 1 + docs/data_driven_config.md | 59 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+) create mode 100644 docs/data_driven_config.md diff --git a/docs/_summary.md b/docs/_summary.md index 19498f6a205b..8d401bb40ce2 100644 --- a/docs/_summary.md +++ b/docs/_summary.md @@ -159,6 +159,7 @@ * [Contributing to QMK](contributing.md) * [Translating the QMK Docs](translating.md) * [Config Options](config_options.md) + * [Data Driven Configuration](data_driven_config.md) * [Make Documentation](getting_started_make_guide.md) * [Documentation Best Practices](documentation_best_practices.md) * [Documentation Templates](documentation_templates.md) diff --git a/docs/data_driven_config.md b/docs/data_driven_config.md new file mode 100644 index 000000000000..7e348ec9e7a6 --- /dev/null +++ b/docs/data_driven_config.md @@ -0,0 +1,59 @@ +# Data Driven Configuration + +This page describes how QMK's data driven JSON configuration system works. It is aimed at developers who want to work on QMK itself. + +## History + +Historically QMK has been configured through a combination of two mechanisms- `rules.mk` and `config.h`. While this worked well when QMK was only a handful of keyboards we've grown to encompass nearly 1500 supported keyboards. That extrapolates out to 6000 configuration files under `keyboard/` alone! The freeform nature of these files and the unique patterns people have used to avoid duplication have made ongoing maintenance a challenge, and a large number of our keyboards follow patterns that are outdated and sometimes harder to understand. + +We have also been working on bringing the power of QMK to people who aren't comformable with a CLI, and other projects such as VIA are working to make using QMK as easy as installing a program. These tools need information about how a keyboard is laid out or what pins and features are available so that users can take full advantage of QMK. We introduced `info.json` as a first step towards this. The QMK API is an effort to combine these 3 sources of information- `config.h`, `rules.mk`, and `info.json`- into a single source of truth that end-user tools can use. + +Now we have support for generating `rules.mk` and `config.h` values from `info.json`, allowing us to have a single source of truth. This will allow us to use automated tooling to maintain keyboards saving a lot of time and maintenance work. + +## Overview + +On the C side of things nothing really changes. When you need to create a new rule or define you follow the same process: + +1. Add it to `docs/config_options.md` +1. Set a default in the appropriate core file +1. Add you `ifdef` and/or `#ifdef` statements as needed + +You will then need to add support for your new configuration to `info.json`. The basic process is: + +1. Add it to the schema in `data/schemas/keyboards.jsonschema` +1. Add code to extract it from `config.h`/`rules.mk` to `lib/python/qmk/info.py` +1. Add code to generate it to one of: + * `lib/python/qmk/cli/generate/config_h.py` + * `lib/python/qmk/cli/generate/rules_mk.py` + +## Adding an option to info.json + +This section describes adding support for a `config.h`/`rules.mk` value to info.json. + +### Add it to the schema + +QMK maintains schema files in `data/schemas`. The values that go into keyboard-specific `info.json` files are kept in `keyboard.jsonschema`. Any value you want to make available to endusers to edit must go in here. + +In some cases you can simply add a new top-level key. Some examples to follow are `keyboard_name`, `maintainer`, `processor`, and `url`. This is appropriate when your option is self-contained and not directly related to other options. In other cases you should group like options together in an `object`. This is particularly true when adding support for a feature. Some examples to follow for this are `indicators`, `matrix_pins`, and `rgblight`. If you are not sure how to integrate your new option(s) [open an issue](https://github.com/qmk/qmk_firmware/issues/new?assignees=&labels=cli%2C+python&template=other_issues.md&title=) or [join #cli on Discord](https://discord.gg/heQPAgy) and start a conversation there. + +### Add code to extract it + +Whenever QMK generates a complete `info.json` it extracts information from `config.h` and `rules.mk`. You will need to add code for your new config value to `lib/python/qmk/info.py`. Typically this means adding a new `_extract_()` function and then calling your function in either `_extract_config_h()` or `_extract_rules_mk()`. + +If you are not sure how to edit this file or are not comfortable with python [open an issue](https://github.com/qmk/qmk_firmware/issues/new?assignees=&labels=cli%2C+python&template=other_issues.md&title=) or [join #cli on Discord](https://discord.gg/heQPAgy) and someone can help you with this part. + +### Add code to generate it + +The final piece of the puzzle is providing your new option to the build system. This is done by generating two files: + +* `.build/obj_/src/info_config.h` +* `.build/obj_/src/rules.mk` + +These two files are generated by the code here: + +* `lib/python/qmk/cli/generate/config_h.py` +* `lib/python/qmk/cli/generate/rules_mk.py` + +For `config.h` values you'll need to write a function for your rule(s) and call that function in `generate_config_h()`. + +If you have a new top-level `info.json` key for `rules.mk` you can simply add your keys to `info_to_rules` at the top of `lib/python/qmk/cli/generate/rules_mk.py`. Otherwise you'll need to create a new if block for your feature in `generate_rules_mk()`. From bd8f3697fa97f4fc99eeb4ef0caba48926d8a449 Mon Sep 17 00:00:00 2001 From: Zach White Date: Sun, 10 Jan 2021 11:58:37 -0800 Subject: [PATCH 2/2] Apply suggestions from code review Co-authored-by: ridingqwerty Co-authored-by: Erovia --- docs/data_driven_config.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/data_driven_config.md b/docs/data_driven_config.md index 7e348ec9e7a6..7e4f2328469d 100644 --- a/docs/data_driven_config.md +++ b/docs/data_driven_config.md @@ -4,7 +4,7 @@ This page describes how QMK's data driven JSON configuration system works. It is ## History -Historically QMK has been configured through a combination of two mechanisms- `rules.mk` and `config.h`. While this worked well when QMK was only a handful of keyboards we've grown to encompass nearly 1500 supported keyboards. That extrapolates out to 6000 configuration files under `keyboard/` alone! The freeform nature of these files and the unique patterns people have used to avoid duplication have made ongoing maintenance a challenge, and a large number of our keyboards follow patterns that are outdated and sometimes harder to understand. +Historically QMK has been configured through a combination of two mechanisms- `rules.mk` and `config.h`. While this worked well when QMK was only a handful of keyboards we've grown to encompass nearly 1500 supported keyboards. That extrapolates out to 6000 configuration files under `keyboards/` alone! The freeform nature of these files and the unique patterns people have used to avoid duplication have made ongoing maintenance a challenge, and a large number of our keyboards follow patterns that are outdated and sometimes harder to understand. We have also been working on bringing the power of QMK to people who aren't comformable with a CLI, and other projects such as VIA are working to make using QMK as easy as installing a program. These tools need information about how a keyboard is laid out or what pins and features are available so that users can take full advantage of QMK. We introduced `info.json` as a first step towards this. The QMK API is an effort to combine these 3 sources of information- `config.h`, `rules.mk`, and `info.json`- into a single source of truth that end-user tools can use. @@ -16,7 +16,7 @@ On the C side of things nothing really changes. When you need to create a new ru 1. Add it to `docs/config_options.md` 1. Set a default in the appropriate core file -1. Add you `ifdef` and/or `#ifdef` statements as needed +1. Add your `ifdef` and/or `#ifdef` statements as needed You will then need to add support for your new configuration to `info.json`. The basic process is: @@ -32,7 +32,7 @@ This section describes adding support for a `config.h`/`rules.mk` value to info. ### Add it to the schema -QMK maintains schema files in `data/schemas`. The values that go into keyboard-specific `info.json` files are kept in `keyboard.jsonschema`. Any value you want to make available to endusers to edit must go in here. +QMK maintains schema files in `data/schemas`. The values that go into keyboard-specific `info.json` files are kept in `keyboard.jsonschema`. Any value you want to make available to end users to edit must go in here. In some cases you can simply add a new top-level key. Some examples to follow are `keyboard_name`, `maintainer`, `processor`, and `url`. This is appropriate when your option is self-contained and not directly related to other options. In other cases you should group like options together in an `object`. This is particularly true when adding support for a feature. Some examples to follow for this are `indicators`, `matrix_pins`, and `rgblight`. If you are not sure how to integrate your new option(s) [open an issue](https://github.com/qmk/qmk_firmware/issues/new?assignees=&labels=cli%2C+python&template=other_issues.md&title=) or [join #cli on Discord](https://discord.gg/heQPAgy) and start a conversation there. @@ -40,7 +40,7 @@ In some cases you can simply add a new top-level key. Some examples to follow ar Whenever QMK generates a complete `info.json` it extracts information from `config.h` and `rules.mk`. You will need to add code for your new config value to `lib/python/qmk/info.py`. Typically this means adding a new `_extract_()` function and then calling your function in either `_extract_config_h()` or `_extract_rules_mk()`. -If you are not sure how to edit this file or are not comfortable with python [open an issue](https://github.com/qmk/qmk_firmware/issues/new?assignees=&labels=cli%2C+python&template=other_issues.md&title=) or [join #cli on Discord](https://discord.gg/heQPAgy) and someone can help you with this part. +If you are not sure how to edit this file or are not comfortable with Python [open an issue](https://github.com/qmk/qmk_firmware/issues/new?assignees=&labels=cli%2C+python&template=other_issues.md&title=) or [join #cli on Discord](https://discord.gg/heQPAgy) and someone can help you with this part. ### Add code to generate it