Reconciliation of Dev and Master

[TerryE]

This was an issue that I proposed opening in #716.

I have looked into doing a git cherry analysis of these to see if we could do a quick reconciliation. Forget this. These branches have been separated for too long and contributions have been almost randomly added to either. In general dev seems to be far more complete and stable than master. So my suggestion of our way ahead in sorting this out is:

• We move master to old-master.
• We create a new baseline master by cloning dev.
• We freeze the new master for a few weeks until we agree a promotion strategy, but in general contributions will only be accepted by committers as PRs into dev.
• In future, contributions will only be promoted into master when they've been proven stable in dev for some proving period, except where there is a consensus amongst committers that there is a tangible benefit in moving it early (e.g. it's an important bugfix). The aim here is that master might not have the latest work in it, but it should at least be stable and usable.
• We will analyse the contributions to old-master that are missing in dev and invite the contributors to cherry pick or roll-up their work as a PR on dev. (See this analysis of differences).

This is only a sketch of a way ahead for feedback and discussion, but I can't think of an easier way of sorting this all out. So comments please.

[jmattsson]

This sounds pretty good to me. The only point where I lean in a different direction is on how to cut releases. I personally favour the approach used by e.g. FreeBSD with a code freeze before a release, where effectively the development tree is given time to get a shake-down and settle/stabilise. If we don't want to slow down things going onto dev, we could easily just branch off dev into an rcXYZ branch which is then frozen, and will only accept bug fixes.

The main advantages of this is approach are

• Releases effectively always come off the development tree, and there is very little divergence between a release tree and the development tree. This helps avoid the place we're currently in.
• It's a very simple process, both to comprehend and to run.
• With the code-freeze people get a gentle poke that focus is on stability for a bit, not on getting awesome-new-feature-Z in.

[marcelstoer]

@jmattsson I pretty much agree.

What I'd like to see clarified though is the notion of "release". With so many available modules now are we ever going to provide pre-built binaries again? If so, which modules are going to be included? The current list of enabled modules seems very random i.e. just as the respective contributor defined it.

[TerryE]

The current list of enabled modules seems very random

As I've mentioned before, my preference would be to spilt out the core firmware from the other modules into separate repositories very much as PHP + PECL and Perl + CPAN is organised. We only have a limited number of base modules in the core, and aa simple way of adding extras. I've been thinking about how this could be implemented, but this is really it's own issue.

This would simplify and facilitate the concept of prebuilt releases as these would only contain core elements, and provide new Lua application developers with a quick start.

The preferred method for users to do their own tailored builds would be to do just that, either locally or through a cloud service.

This would also facilitate Johnny's suggested release approach, which has a lot of merit, because we're only be freezing the core.

[jmattsson]

I've been toying with the idea of implementing loadable C modules but no matter which way I look at it, it opens up a serious can of worms around dependencies, versioning and upgradeability. In the end I keep coming back to something like Marcel's cloud builder being where the sweet spot is.

I can't say I'm too keen on splitting out modules into their own repo, since they're by necessity very closely tied to the base system & especially the SDK. Again, I foresee a serious dependency/versioning headache in that direction.

[TerryE]

I've also though about doing C loadables and it just doesn't work. If we had a standard host side scripting language (Lua 5.1??) then some bundled package process would be easily doable, but this would require either (i) the user has a host system/ container / VM running Lunix plus the toolchain or (ii) we extend Marcel's model.

I think that power users will plump for the former but the average Lua esp8266 developer will use cloud service.

[TerryE]

I've been going through these changes and the situation is a lot better than it first appeared. My main problem is that we could really do with better review of these examples before accepting them. Still I've fixed one and resolve a few, pushing them back to their authors. only a few to go.

[jmattsson]

Thanks Terry!

[TerryE]

I missed the fact that Vowstar quite often committed changes into master directly without uploading them first to his own fork and then issuing a PR. I am going through these as well. 😞

[jmattsson]

We're seeing a lot of people still trying to use the old master branch. Can we fast-track the master->oldmaster and snapshot dev into a new master branch? At this point I'd say that not having all of the oldmaster fixes across into newmaster is the lesser evil.

[nickandrew]

Agreed, the lesser evil.

[TerryE]

+1 I can do this tomorrow, but let us give this another day for any committers or significant contributers to lodge a counter argument. 😄

[nodemcu]

+1

[TerryE]

OK, master -> old-master
dev has Marcel's "Remove pre-builts" patch applied
dev copied -> master
@marcelstoer Marcel, can you do a check on Frightannic's build of master?

I still have a couple of old-master patches to apply to dev and master, and then I'll close this issue.

[marcelstoer]

@TerryE Looking good over here.

[nickandrew]

Good stuff.

Right now would be an excellent time to merge my PR #769 on the README.md - while not perfect, it is at least up-to-date for 1.4.0.

[TerryE]

Done.