-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC: modify buildsystem docs in README.md #825
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,28 +1,36 @@ | ||
# Freifunk Berlin Firmware | ||
https://wiki.freifunk.net/Berlin:Firmware | ||
|
||
This is the build-system for the Firmware of Freifunk Berlin. | ||
The firmware is based on vanilla [OpenWrt](https://openwrt.org/start) with some modifications (to fix | ||
broken stuff in OpenWrt itself or for example LuCI) and additional default packages/configuration settings. | ||
This is the build-system for the Firmware of Freifunk Berlin. The firmware is | ||
based on vanilla [OpenWrt](https://openwrt.org/start) with some modifications | ||
(to fix broken stuff in OpenWrt itself or for example LuCI) and additional | ||
default packages/configuration settings. | ||
|
||
|
||
## Contact / More information | ||
|
||
More user relevant information about the firmware are on the wiki page at: https://wiki.freifunk.net/Berlin:Firmware. There you can also find the | ||
* [ReleaseNotes](https://wiki.freifunk.net/Berlin:Firmware/v1.0.2) | ||
* a tutorial ([en](https://wiki.freifunk.net/Berlin:Firmware:En:Howto) / [de](https://wiki.freifunk.net/Berlin:Firmware:Howto)) on router configuration | ||
More user relevant information about the firmware are on the wiki page at: https://wiki.freifunk.net/Berlin:Firmware. | ||
There you can also find the | ||
* [ReleaseNotes](https://wiki.freifunk.net/Berlin:Firmware/v1.0.3) | ||
* a tutorial ([en](https://wiki.freifunk.net/Berlin:Firmware/en/HowTo) / [de](https://wiki.freifunk.net/Berlin:Firmware/HowTo)) on router configuration | ||
|
||
For questions write a mail to <[email protected]> or come to our | ||
weekly meetings. If you find bugs please report them at: https://github.com/freifunk-berlin/firmware/issues | ||
|
||
For questions write a mail to <[email protected]> or come to our weekly meetings. | ||
If you find bugs please report them at: https://github.com/freifunk-berlin/firmware/issues | ||
|
||
## Development | ||
|
||
### Info | ||
|
||
For the Berlin Freifunk firmware we use vanilla OpenWrt with additional patches | ||
and packages. The Makefile automates firmware | ||
creation and apply patches / integrates custom freifunk packages. All custom | ||
patches are located in *patches/* and all additional packages can be found at | ||
http://github.com/freifunk-berlin/packages_berlin. | ||
How does this stuff work? | ||
|
||
The idea is to download OpenWrt via git, patch it with all patches in the | ||
`patches/` folder and configure it according to the configuration snippets in | ||
`configs`. Then use the OpenWrt buildsystem to build all packages and the | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. should be configs/ , as it's a folder |
||
Imagebuilder and use the latter to assemble the final image with lists of | ||
packages in packages for the router. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "with lists of packages in packaglists/ for the router. |
||
All additional packages can be found at http://github.com/freifunk-berlin/packages_berlin. | ||
Akira25 marked this conversation as resolved.
Show resolved
Hide resolved
Akira25 marked this conversation as resolved.
Show resolved
Hide resolved
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. should we mention "gluon-packages" and "freifunk-potsdam-packages" also. (not shure if ffp-packages made it up to this repo already) |
||
|
||
|
||
### Build Prerequisites | ||
|
||
|
@@ -48,6 +56,41 @@ pacman -S base-devel git ncurses lib32-zlib gawk time unzip perl-xml-libxml \ | |
flex wget gettext quilt python2 openssl | ||
``` | ||
|
||
|
||
### Build-System-Structure | ||
|
||
Where can I change something? What does these files do? | ||
Akira25 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
``` | ||
:: Settings | ||
|
||
config.mk - OpenWrt git repository, revision, select what package lists to build | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "OpenWrt git repository, revision" is controlled by modules-file |
||
feeds.conf - OpenWrt feeds.conf that is used for the package feeds during build | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "feeds.conf" is converted to modules |
||
|
||
configs/ - further configurations | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "further configurations" replace by "snippets of the OpenWrt config-options (compare to call "make menuconfig" in openwrt/)" or something alike |
||
common.config - OpenWrt config-parameters for all targets | ||
$(TARGET).config - target-specific config. Gets added to common.config. Options from target.config | ||
overwrite those from common.config. | ||
common-autobuild.config - gets added on top , if IS_BUILDBOT=yes is defined | ||
|
||
|
||
patches/ - Patches against the OpenWrt directory | ||
packagelists/ - Package Lists | ||
profiles/ - List of router profiles for each architecture - names come from the Imagebuilder | ||
|
||
Akira25 marked this conversation as resolved.
Show resolved
Hide resolved
Akira25 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
:: Building | ||
|
||
Makefile - Does all the stuff. Cloning and Updating OpenWrt, patching, running make | ||
assemble_firmware.sh - Script for running the Imagebuilder for a target | ||
|
||
|
||
:: Images | ||
|
||
firmwares/ - Folder where the final images end up | ||
|
||
``` | ||
|
||
|
||
### Building all firmwares | ||
|
||
To get the source and build the firmware locally use: | ||
|
@@ -66,17 +109,18 @@ connection can improve the build time. | |
|
||
You need approximately 10GB of space for the build. | ||
|
||
### Directory Layout | ||
|
||
You can find the actual firmware images generated by the ImageBuilder (and the ImageBuilder itself) | ||
in `firmwares`. The layout looks like the following: | ||
### Directory Layout of `firmwares/` | ||
|
||
You can find the actual firmware images generated by the ImageBuilder (and the | ||
ImageBuilder itself) in `firmwares`. The layout looks like the following: | ||
|
||
``` | ||
firmwares/ | ||
TARGET/ | ||
backbone/ | ||
manual/ | ||
images.. | ||
default/ | ||
notunnel/ | ||
images.. | ||
... | ||
OpenWrt-ImageBuilder-....tar.xz | ||
|
@@ -94,73 +138,103 @@ firmwares/ | |
*.ipk | ||
``` | ||
|
||
As you notice there are several different image variants ("backbone", "default", etc.). | ||
These different *packages lists* are defined in `packages/`. | ||
See the "Features" section above for a description of the purpose of each package list. | ||
With the "OpenWrt-Imagebuilder" you can assemble your own image variant with your | ||
*packages lists* without having to compile everything yourself. The "OpenWrt-SDK" is | ||
the fastest way to build your own packages or programs without compiling OpenWrt itself. | ||
The "initrd" directory contains some initrd-images for netboot, which are required on | ||
some boards to initially install OpenWrt. | ||
As you notice there are several different image variants ("manual", "notunnel", | ||
etc.). | ||
+ These different *packages lists* are defined in `packages/`. See the | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You talk here on the "firmware" folder, right? Then this packages folder is just the storage-repo of all packages available.These packages will be included according to the flavors in packagelists/ to the image or used as source then running opkg install on the router. |
||
"Features" section above for a description of the purpose of each package list. | ||
+ With the "OpenWrt-Imagebuilder" you can assemble your own image variant with | ||
your *packages lists* without having to compile everything yourself. The | ||
*OpenWrt-SDK* is the fastest way to build your own packages or programs without | ||
compiling OpenWrt itself. | ||
+ The "initrd" directory contains some initrd-images for netboot, which are | ||
required on some boards to initially install OpenWrt. | ||
|
||
|
||
### customizing make | ||
|
||
`make` will use by default `TARGET` and `PACKAGES_LIST_DEFAULT` defined in | ||
`config.mk`. You can customize this by overriding them: | ||
|
||
``` | ||
make TARGET=mpc85xx PACKAGES_LIST_DEFAULT=backbone | ||
make TARGET=mpc85xx-generic PACKAGES_LIST_DEFAULT=manual | ||
``` | ||
in addition you can build your own image from a prebuilt imagebuilder by something like: | ||
|
||
``` | ||
make images IB_FILE=<file> TARGET=... PACKAGES_LIST_DEFAULT=... | ||
``` | ||
|
||
The default target is `ar71xx-generic`. For a complete list of supported targets look in `configs/` for the target-specific configs. | ||
Each of these targets need a matching file in `profiles/` with the profiles (boards) that should be build with the imagebuilder. | ||
The default target is `ar71xx-generic`. For a complete list of supported targets | ||
look in `configs/` for the target-specific configs. Each of these targets need a | ||
matching file in `profiles/` with the profiles (boards) that should be build | ||
with the imagebuilder. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "should be build by the imagebuilder"? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "should be built by" would be the queen's english. |
||
|
||
additional options | ||
|
||
* IS_BUILDBOT : | ||
* this will be "yes" when running on the buildbot farm and helps to save some disc-space by removing files not required anymore. On manual builds you should not set this to "yes", as you have to rebuild the whole toolchain each time. | ||
* this will be "yes" when running on the buildbot farm and helps to save some | ||
disc-space by removing files not required anymore. On manual builds you | ||
should not set this to "yes", as you have to rebuild the whole toolchain each | ||
time. | ||
* SET_BUILDBOT : | ||
* "env" the Makefile will honor the "IS_BUILDBOT" environment | ||
* "yes" the Makefile will always act as "IS_BUILDBOT" was set to "yes" | ||
* "no" the Makefile will always act as "IS_BUILDBOT" was set to "no" / is unset. This way we can run builds on the buildbot like a local build. | ||
* "no" the Makefile will always act as "IS_BUILDBOT" was set to "no" / is unset. | ||
This way we can run builds on the buildbot like a local build. | ||
|
||
|
||
### Build a new router model for an existing architecture | ||
|
||
Everything that is in the [Table of Hardware](https://openwrt.org/toh/start) | ||
should work at least in theory. Wifi should be ath9k / ath10k. mt76 also appears | ||
to work. | ||
|
||
You can use the Imagebuilder to get a list of all supported models for a | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Probably the better place, than the mentioned above, to reference "make info" if we want to reference it at all. |
||
platform. You can also look in the [source](https://git.openwrt.org/?p=openwrt/openwrt.git;a=tree;f=target/linux/ar71xx/image) | ||
for model names. | ||
|
||
Just put the name in the `/profiles/<target>.profiles` and it should work. | ||
Akira25 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Akira25 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
### Continuous integration / Buildbot | ||
|
||
The firmware is [built | ||
automatically](http://buildbot.berlin.freifunk.net/one_line_per_build) by our [buildbot farm](http://buildbot.berlin.freifunk.net/buildslaves). If you have a bit of CPU+RAM+storage capacity on one of your servers, you can provide a buildbot slave (see [berlin-buildbot](https://github.com/freifunk/berlin-buildbot)). | ||
The firmware is [built automatically](http://buildbot.berlin.freifunk.net/one_line_per_build) | ||
by our [buildbot farm](http://buildbot.berlin.freifunk.net/buildslaves). If you | ||
have a bit of CPU+RAM+storage capacity on one of your servers, you can provide | ||
a buildbot slave (see [berlin-buildbot](https://github.com/freifunk/berlin-buildbot)). | ||
|
||
All branches whose name complies to the "X.Y.Z" pattern are built and put into | ||
the "stable" downloads directory: [http://buildbot.berlin.freifunk.net/buildbot/stable/](http://buildbot.berlin.freifunk.net/buildbot/stable/) | ||
|
||
All branches whose name complies to the "X.Y.Z" pattern are built and put into the "stable" downloads directory: | ||
[http://buildbot.berlin.freifunk.net/buildbot/stable/](http://buildbot.berlin.freifunk.net/buildbot/stable/) | ||
All branches with names not fitting the "X.Y.Z" pattern are built and put into | ||
the "unstable" directory: [http://buildbot.berlin.freifunk.net/buildbot/unstable/](http://buildbot.berlin.freifunk.net/buildbot/unstable/) | ||
Note that in the directory there is no reference to the branch name; unstable | ||
builds can be identified by build number only. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. referencing the VERSION.txt file? As this is the best place to look up on which branch and revision a build is based on. |
||
|
||
All branches with names not fitting the "X.Y.Z" pattern are built and put into the "unstable" directory: | ||
[http://buildbot.berlin.freifunk.net/buildbot/unstable/](http://buildbot.berlin.freifunk.net/buildbot/unstable/) | ||
Note that in the directory there is no reference to the branch name; unstable builds can be identified by build number only. | ||
|
||
#### Creating a release | ||
|
||
Every release has a [semantic version number](http://semver.org); each major version has its own codename. | ||
We name our releases after important female computer scientists, hackers, etc. | ||
For inspiration please take a look at the related | ||
[ticket](https://github.com/freifunk-berlin/firmware/issues/24). | ||
Every release has a [semantic version number](http://semver.org); each major | ||
version has its own codename. We name our releases after important female | ||
computer scientists, hackers, etc. For inspiration please take a look at the | ||
related [ticket](https://github.com/freifunk-berlin/firmware/issues/24). | ||
|
||
For a new release, create a new branch. The branch name must be a semantic version | ||
number. Make sure you change the semantic version number and, for major releases, | ||
the codename in the README and config files (./configs/*) | ||
For a new release, create a new branch. The branch name must be a semantic | ||
version number. Make sure you change the semantic version number and, for major | ||
releases, the codename in the README and config files (`./configs/`) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ( |
||
|
||
The buildbot will build the release and place the files in the stable direcotry | ||
once you pushed the new branch to github. | ||
|
||
|
||
### Patches with "git format-patch" | ||
|
||
**Important:** all patches should be pushed upstream! | ||
|
||
If a patch is not yet included upstream, it can be placed in the corresponding subdirectory below the`patches` | ||
directory. To create a correct patch-file just use the [`git format-patch`](https://git-scm.com/docs/git-format-patch) command. | ||
If a patch is not yet included upstream, it can be placed in the corresponding | ||
subdirectory below the`patches/` directory. To create a correct patch-file just | ||
use the [`git format-patch`](https://git-scm.com/docs/git-format-patch) command. | ||
|
||
|
||
#### Create a patch | ||
|
||
|
@@ -181,21 +255,23 @@ or continue to the relevant feed directory: | |
cd feeds/luci | ||
``` | ||
|
||
use the normal `git commit` workflow to apply your changes to the code. When done convert your last commit | ||
into a patch by running: | ||
use the normal `git commit` workflow to apply your changes to the code. When | ||
done convert your last commit into a patch by running: | ||
|
||
```bash | ||
git format-patch --start-number <n> HEAD^ | ||
``` | ||
where `n` is the next free number of the correlating patch-subdirectory. You can use something like `HEAD^^^^` | ||
to create patch-files from you last 4 commmits, or even use a git-rev directly. Feel free to squash multiple | ||
commits into a single one before creating the patch-file or use something like | ||
where `n` is the next free number of the correlating patch-subdirectory. You can | ||
use something like `HEAD^^^^` to create patch-files from you last 4 commmits, or | ||
even use a git-rev directly. Feel free to squash multiple commits into a single | ||
one before creating the patch-file or use something like | ||
|
||
```bash | ||
git format-patch --stdout HEAD^^^^ > patches/routing/0008-awesome.patch | ||
``` | ||
to create a single file of these 4 commits | ||
|
||
|
||
#### Modify a patch | ||
|
||
To update an existing patch do the same as above: | ||
|
@@ -205,31 +281,35 @@ make clean patch | |
cd openwrt | ||
cd feeds/luci | ||
``` | ||
Then just add a new commit with your changes and squash it with the commit relating to the patch-file. | ||
To update the patch-file use the same `git format-patch` sequence as you did when creating the patch | ||
initially. | ||
Then just add a new commit with your changes and squash it with the commit | ||
relating to the patch-file. To update the patch-file use the same | ||
`git format-patch` sequence as you did when creating the patch initially. | ||
|
||
|
||
#### Delete a patch | ||
|
||
To remove a patch-file you have to remove it from the patch-subdirectory and update the build- | ||
environment: | ||
To remove a patch-file you have to remove it from the patch-subdirectory and | ||
update the build- environment: | ||
|
||
```bash | ||
git rm patches/openwrt/0010-unrelevant-change.patch | ||
make patch | ||
``` | ||
|
||
|
||
### Submitting patches | ||
|
||
#### Freifunk Berlin | ||
|
||
Please create a pull request for the project you want to submit a patch. | ||
If you are already member of the Freifunk Berlin team, please delete branches once they have been merged. | ||
Please create a pull request for the project you want to submit a patch. If you | ||
are already member of the Freifunk Berlin team, please delete branches once they | ||
have been merged. | ||
|
||
|
||
#### OpenWrt | ||
|
||
Create a commit in the openwrt directory that contains your change. Use `git | ||
format-patch` to create a patch: | ||
Create a commit in the openwrt directory that contains your change. Use | ||
`git format-patch` to create a patch: | ||
|
||
``` | ||
git format-patch origin | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To reflect the current state: It feels to me, that we should not reference the wiki here, as nobody takes care of updating the pages. It seems best to reference the Github-release pages (https://github.com/freifunk-berlin/firmware/releases) here, which link to the correct ReleaseNotes of the corresponding release.