build&test openwrt packages

[markus2330]

We should try to build

https://github.com/openwrt/packages/blob/master/libs/elektra/Makefile

on our CI.

later steps:

• also build with BUILD_TESTING=ON and let the tests run on some hardware.

[kodebach]

Why should we test an external package on our CI??

The openwrt/packages does have a CI setup, so the package should really be tested in their CI (AFAICT it is).

[markus2330]

Why should we test an external package on our CI??

So that we find problems before the release (and not after).

[kodebach]

But it is an external package, why do we (the maintainers of Elektra, but not of this package) care whether it works?

Also what is special about this package? If we test this package to make sure we don't break it with a release, why don't we test other external packages too? Where is the line of what we should and shouldn't test on our CI? Normally, you just test the things within your own repo to avoid exactly this problem.

[markus2330]

But it is an external package, why do we (the maintainers of Elektra, but not of this package) care whether it works?

Source-only releases are not sufficient in the early-adapters phase we are in. We need to avoid situations like #3682

Also what is special about this package?

It is the only one that uses:

1. cross compilation cross compile Rust #3904
2. in-source-tree builds disable support for in-source-tree builds #3451
3. ARM
4. non-(eg)libc

So it vastly improves the scope of which types of bugs we could find during development.

why don't we test other external packages too?

We plan to add some others, see #3519.

Where is the line of what we should and shouldn't test on our CI?

There is no line, there is simply a limited amount of resources with which we optimize user satisfaction.

[kodebach]

4. non-libc

Do you mean glibc? I don't see how Elektra would work without any libc...

So it vastly improves the scope of which types of bugs we could find during development.

Another way to look at it is: It vastly expands the scope we need to suppor, i.e. getting to a stable 1.0 would be much simpler with a more limited scope.

We plan to add some others

Okay, then I'm fine with this, as long as there is some sort of process to determine what we build on our CI. Building just some random packages without first thinking about the benefits for Elektra, would quite literally be a waste of resources.

I would also strongly suggest, that builds of external packages should have a much lower priority on the Jenkins Server. Meaning, the build speed of libelektra should never be affected by builds for external packages. It would be very annoying, if I have to wait for some external package build to finish before I get feedback for my libelektra PR.

[markus2330]

Do you mean glibc?

Exactly, the package is called libc6 in Debian, though.

Another way to look at it is: It vastly expands the scope we need to suppor, i.e. getting to a stable 1.0 would be much simpler with a more limited scope.

The scope is according to doc/WHO.md

I would also strongly suggest, that builds of external packages should have a much lower priority on the Jenkins Server.

Yes, I totally agree. In #3519 we already planned to have a separated pipeline for long-running tests.

[markus2330]

basically we need to get the openwrt toolchain (See also #3904), copy the Elektra Makefile to package/libs/elektra and run make || make -j1 V=sc

[markus2330]

My router has a MIPS 24Kc V7.4 CPU. But it is probably not worth building the packages just for my router.

@haraldg do you only use arm or do you also target another platform?

[haraldg]

I personally only use OpenWRT with arm_arm926ej-s/mxs however elektra automatically gets built by OpenWRT on all platforms. If your router has OpenWRT then you should be able to download a proper binary package from downloads.openwrt.org

BTW: OpenWRT Makefiles can fetch any git commit by hash from a git repository as source. Building some script, that just generates a custom Makefile for your CI test should be trivial.

However, maintaining such a package Makefile to keep track of all the changes in elektra (added/removed files etc) is basically the some effort as maintaining the package itself. If you go that route, you might as well adopt the package in OpenWRT. If you - understandably - don't want to put in that effort, then probably it is better to just have the SDK installed somewhere to be able to reproduce issues when we find them in OpenWRT.

[markus2330]

We added openwrt x86 builds to our build server, see #3967

[mpranj]

@robaerd an error appeared on master with the openwrt builds: https://build.libelektra.org/blue/organizations/jenkins/libelektra/detail/master/581/pipeline/2617
Can you take a look if this is temporary or we need to fix something?

[robaerd]

I could reproduce the issue and it also seems to appear for other packages on the CI of openwrt/packages for x86_x64 builds as well, see https://github.com/openwrt/packages/runs/3773018124 or https://github.com/openwrt/packages/runs/3777823383?check_suite_focus=true.

@haraldg do you maybe know why this error is happening?

[haraldg]

I don't know anything in this case. But in my experience it's not uncommon for random packages of openwrt to break on minor updates and such. I guess, given that they are one of very few folks to systematically cross compile a huge collection of software, they just catch an unproportional number of build system related regressions.