Creating a new workflow for spec work is confusing and counterproductive.

[ara4n]

@maxidor Rather than building a forked version of the Matrix spec with its own URL prefixes, workflow and governance process, please can we work together to improve the official spec directly - otherwise you risk confusing and fragmenting the spec process even further, as well as creating an artificial and unconstructive distinction between the community & the core team. There should be only one Matrix community, of which the core Matrix.org team happens to be a subset.

The intended model for contributing to the Matrix spec (expanded out from matrix-org/matrix-doc's CONTRIBUTING.rst) is:

• Write a draft of the proposed or missing content in Google Docs. We originally started off trying to use the /drafts folder in Github and PRing against it, but it ended up being disasterous - e.g. messes like Straw-man HTML messages proposal matrix-org/matrix-spec-proposals#92 and Attaching data to messages matrix-org/matrix-spec-proposals#93, as GH doesn't provide a sensible UI for viewing historical comments or tracking documentation (versus code) revisions or proposed changes. Since moving to Google Docs we haven't looked back, and it would be a huge backwards step to start doing PRs against markdown in /drafts again.
• Publicise & discuss the draft in #matrix-dev:matrix.org or perhaps #matrix-architecture:matrix.org (although matrix-architecture's signal-to-noise ratio doesn't seem to be great) to gather feedback from the community, including the core team. Actual concrete feedback and QA happens in comments on the googledocs.
• The doc is added to the shared spec drafts folder on Google Drive
• File an issue against matrix-org/matrix-doc to track the existence of the feature/fix proposed by draft and its overall progress
• Get some feedback from the community (which includes the core team) on the draft
• Implement the proposal to demonstrate if/how it works, using the /_matrix/$api/unstable API prefix, PRing them at least against one server or client implementation as appropriate.
• Once there's some consensus on the discussion on the draft and the implementation PRs are accepted and seen to work, submit a PR against matrix-org/matrix-doc.
• Folks with commit access to matrix-org/matrix-doc do a final review on the PR and merge it.
• At the next release of the spec, the /unstable API prefix gets released as a /rx.y.z API. (Yes, I know we are horrifically behind on releasing specs, but this should be improved in the near future once we manage to locate someone trusted to manage the review/merge/release process).

So far we have merged almost all PRs against matrix-org/matrix-doc, and the google-doc based review model has proved successful, so I do not understand why a separate process is needed here. The concerns I have are:

• The README for this repository proposes a new workflow which duplicates the existing one, without explaining the issues that it's trying to address. Instead, it would be better to surely improve the existing process rather than create a parallel system.
• It creates and encourages an artificial divide between the "community" and the "core team". This divide does not exist: there is only one Matrix community, and we should all be contributing to the same project rather than forking or fragmenting it.
• Creating a new URL prefix (/_evt-matrix) makes the existing URL prefix situation even more confusing, and is tantamount to fragmenting the standard - something we are desperately trying to avoid to do, to avoid an XMPP-like situation.
• It unilaterally proposes a new governance process (e.g. "Eventually Matrix will also provide a process for gaining community consensus to attempt to move these spec changes into the official matrix-org spec.") rather than working to improve the current one.
• It duplicates effort (in terms of spec management) and is going to make it even harder to track and avoid redundant/duplicated spec work.

Now, of course, there is nothing stopping you from forking and fragmenting the protocol, giving it your own namespace and creating your own variant. However, it should be pretty obvious that creating schisms in the project is very unlikely to be helpful, and I would implore you to try to understand and participate in and improve the existing workflow instead. (cc @non-Jedi).

[ara4n]

(A good example of how this duplicate/overlapping process is already introducing confusion and obscuring privacy issues can be found at https://github.com/eventually-matrix/matrix-doc/commit/9e71fd668972605ed98b4b275f10b4185bd9417a).

[non-Jedi]

@ara4n Thanks for raising your concerns. Will take me some time to read
through and think about my response. Just wanted to say right off the bat
that this is definitely not a project to fork the spec (well, other than
in a trivial sense); the intention and hope for everything we do here is to
improve the official matrix.org spec. I'll definitely reevaluate both
approach and whether what we're doing is a positive at all based on your
comments.

[maxidorius]

@ara4n Before going into a detailed reply, I would like to state a few things and use those as a foundation for the rest.

• This is my answer and my answer only. It is a pragmatic one. I will talk about current facts and state of affairs, irrelevant of how they came to be. I will also expose my feelings towards those facts while I acknowledge and respect your feelings that we are needlessly diverting work and splitting up the community.
• I never thought of stopping doing PRs against matrix.org repos and I will continue to try working with the whole community to improve the protocol. Any difference between the matrix-doc repos are only what has not been accepted upstream.
• Matrix was a direct hit for me, because it is the only open protocol to deal with the IM needs. My interest has never been about synapse or Riot or sydent. While I had to deal with them to understand some things, my interest lies with the protocol. It has only one deliverable: the specification.
• I have one goal: be able to create software which can talks to the current Matrix ecosystem to answer the needs of my company and those of my clients.
• I will use examples to illustrate my points, but discussing the specific about those examples should be done somewhere else. I believe we can resolve any conflict in those if taken individually using a more suited discussion medium.

After I came into contact with Matrix in January 2017, I quickly saw its potential and wanted to get to know the protocol better, and see how implementations could be customized/created to fit my needs. I quickly came to the conclusion that:

• The spec was way outdated:
  • No new stable release for the CS API since July 2016 (~6 months), HEAD was missing lots of new features.
  • No stable release at all for anything else, and what existed was simply not usable.
• The only implementations that had any value for reverse engineering were the Matrix.org ones. Nobody else managed to come close to synapse/Riot, most likely for a long time.
• For better or worse, synapse and Riot have a monopoly. Whatever you decide to implement, if it doesn't comply with those, you're cutting yourself off from the Matrix network.
  Later, it was made clear we shouldn't even bother with the spec, but instead directly deal with implementations and then put our findings into the unstable spec (master branch).

We have been in contact since January, I have created several tools and made several contributions, trying my best to respect your wishes that the spec was not high priority and worked in my own corner, proudly showing my progress from time to time. Then the funding crsis came. None of the issues that existed at least 1 year prior to it were addressed at that point, which only made them worse.

During that time I have met several people which have voiced the same concerns as I have concerning the effective closing of the Matrix protocol due to a lack of documentation, making it very difficult to create working implementations. (Ruma is an example of a promising project which has been in limbo because of it)
Some people do not want to share those in public, in fear of loosing privileged status or recognition with the core team, preferring to keep discussion behind closed doors.
Others believe communication is key and discussions behind closed doors is not the way forward, and that everyone in the Matrix community should have a decent chance to give feedback.

I am now at a point where I have understood parts of the protocol and implementations enough and validated those with my own implementations to share this acquired knowledge with the community.
My belief is also that Matrix can only grow further by allowing new implementations to exists, challenging the spec and other implementations, forcing a constant improvement.

I have tried (most likely in a clumsy way, given the responses I got) to contribute this knowledge about how current implementations work, but I have hit a wall using various channels and felt I was only a bother to the core team.
You have stated several times the spec is not high priority and that you are not willing to waste more of the core team time on some of the discussions we had in the past.
This has lead to several clashes between us, which always made me feel this should have been done a different way. While there was definitely successful PRs (a huge thanks to Rich btw) for trivial items, there hasn't been a successful "difficult" PR so far.

Now to address your specific points:
While the spec contribution seems to work fine for the core team and for other relatively trivial changes, I don't believe it works with the community for several reasons:

• Most people don't have time to go through the whole process. I would think raw or in spec format (not necessarily spec quality) documentation, even if incomplete, would be better than nothing.
  Several PRs were open to simply improve the latest master in an incremental process. Those interactions were unsatisfactory in the end, as the reviews were always sidetracked into (this is my own interpretation of it) "we want better, do it for us", effectively burning out the contributor and loosing that information, instead of accepting it and improving it in the way you feel is best.
  Let's keep in mind PRs are done against master, the unstable HEAD version of the spec - effectively super bleeding edge.
• There are many points which are tribal knowledge within core team and were not shared publicly at all or in a visible manner.
  There is also a lot of "common sense" involved. Overall, it's almost impossible to produce a non-trivial PR against the spec and have it merged.
  There usually is something to fix which puts a big burden on a community member which is spending their free time on this.
• The overall acceptance level for spec PR is very high, too high I believe, with the current system. If the non-core community was to bring knowledge as they have it and the core team would follow-up on required changes after the PR was accepted, it would make the whole process much more welcoming.
• You have PRs open since 2016 that only require trivial changes to be accepted, and even one which seems ready to go but had absolutely no feedback.
  This is sending mixed signals between your words of wanting PRs badly and your actions of being so passive/demanding about them.
• Just like you, I believe implementations should come first to prove that the change works. Some people (like me) don't believe in using Google docs to share the API description of a new proposal works tho.
  If you want to validate an implementation, you should do so with as many people as you can find, not with just your own.
  It is much easier to agree on what to implement in the OpenAPI format than a Google Doc, which is always unprecise.
• Due to the tendancy of breaking stable/unstable namespaces by stable releases of Riot/synapse, we cannot be safe in using it for any attempt to create a new feature, or improve an existing one as it could break existing stable implementations.
  There is also no will to make reference implementations compliant ASAP, making community efforts difficult.
• Finally, the biggest pain point, the core devs don't follow the spec and have no problem breaking it on a regular basis. This means that the work done by the community can be pointless at any point, making the whole contributing experience rather frustrating. It's hard to have any motivation doing work which will not be used anyway.

Why I believe we need a complementary process:

• Purposefully or not, true or not, and while I know better, it feels like the community is only an after-thought of the core devs. Best examples are how there was no public discussion/reviews about Widgets, Groups and Directory lookups.
• The core team is and always was a huge bottleneck, for good reasons. We can only admire your passion and see how you are overwhelmed by the sheer amount of work.
  Spec work should only go through you when a contribution is at its finest, not taking more time than necessary.
  So instead of wasting your time and debate for hours, we took on rich advises [1] [2] to do it in our own repo so we can present either raw PRs, which you are free to accept or refuse as is, or curated PRs which are worth discussing about.
• We want to document the current behavior of implementations in a spec format (even if not necessarily spec quality) for ease of PRs, for ease to implementations accross several 3rd parties.
  I have asked how to do this to you in DM (without any answer) and in public rooms. The clear answers we got [1] [2] were that it is not seen as something useful.
  This has made it clear our goals are not the same and there was no way to currently reach a good outcome if we kept talking about it.
• We keep getting mixed signals from various core members (happy to discuss specific examples in a more suited place). We also tried to reach you personally several times for clarification (in DMs, in public rooms pointing to such DMs, just in public rooms, in the weekly sync) but due to the current situation, you have very much other issues to deal with, rightfully so.
  This usually ended up in a clash were nothing productive could have some out of it, or simply DMs being ignored/not seen.

To conclude:

• This is about documenting existing behaviour since so far our PRs to do so cannot be hosted by the core team. [1].
• I would be so happy if we could directly work with the core devs and truly improve the specification in an incremental way, which aligns with the available time and means of contributors.
• Whatever happens in this discussion will not change anything to my dedication to Matrix, to my monthly donations, to my PRs towards the spec. I WILL keep helping in a way that I believe profits to the Matrix protocol.
• I feel burned out trying to simply add knowledge somewhere and constantly feel like I'm having to justify documenting the current implementations when only those have any value. My interactions with other people make me believe I am not alone.
• There is a real frustration seeing the constant conflict of interests of the core team between Matrix as a protocol and the reference implementations.
  Like many protocols and specifications out there, this should be backed by The Community (at large) and prominent figures within the communities should have an active part in the process of doing the spec.
  The core team should also bite the bullet and fix their implementations when not following such spec so implementations can actually co-exist.
• I strongly believe creating this repo is useful as constant reverse-engineering for differences and lacking a single self-contained document is a huge waste of time for anyone trying to implement Matrix.
  The sooner there is a full documentation of the protocol reflecting the current state, the best. I don't ask for a stable release, nor do I ask for any implementations to follow it to the letter.
  I only ask for a reference document which is as complete/accurate as possible where errors or sudden changes are acceptable. My ideal view: A draft of the spec, representing the latest state of affairs, in spec format.

[erikjohnston]

First of all, let me just say that we really do appreciate the enthusiasm and dedication from everyone here, and we really do want to try and find a way forwards that works for all of us.

The core team has talked through the issues raised here today, and the rest of this post is a reflection of that discussion on behalf of all of us.

@maxidor has raised lots of points, some of them are perfectly understandable, some of them we disagree with, and some of them are due to us miscommunicating.

The situation with the spec is currently… a mess. It’s outdated, PR reviews sometimes get stuck, and basically it’s been unloved for too long. This is frustrating for everyone involved, especially us, and I admit that we can get a bit defensive on that front.
Due to understaffing of the core team we’ve been unable to find the right person to help really push all of our documentation forwards; however, it is high on our list of priorities when the funding situation has been resolved (and in fact we have some candidates lined up already). Until then, all we can do is try and prioritise what we think will help the project the most right now, and that sadly often means that the documentation doesn’t get the love it deserves.

Having said that, you’ve raised a number of points about how we handle the spec PRs when we do have time to look at them:

The overall acceptance level for spec PR is very high ... I would think raw or in spec format (not necessarily spec quality) documentation, even if incomplete, would be better than nothing

This may very well be fair; it may be worth discussing if there is a way for incremental changes to be accepted without them immediately landing─some sort of staging area perhaps. It does feel that contributing to the spec is perhaps harder than it should be at the moment. Certainly looking into this would be a priority for anyone we got working on the spec, if not before.

It is worth noting your recent PRs, e.g. #1039 are not trivial changes (even if they don’t require implementation changes), and so don’t fall into this category.

In general we follow the standard FOSS philosophy of asking contributors to make revisions to their PRs; this helps contributors learn from the comments and reduces the burden with accepting contributions. While this works well for us in our code repos, it may not be the most appropriate way of accepting contributions to the spec.

Due to the tendancy of breaking stable/unstable namespaces by stable releases of Riot/synapse … Finally, the biggest pain point, the core devs don't follow the spec and have no problem breaking it on a regular basis.

I know that @richvdh has said this, but I think it's a misrepresentation of what actually happens:

• We do add APIs and extra functionality all the time, but in backwards compatible ways (though mistakes are sometimes made).
• Very occasionally we do make backwards incompatible changes. This is done with care, e.g. when an API is broken so we know no one can be using it, or due to security vulnerabilities, etc. For example, recently the /tokenrefresh API was removed due to the fact that refresh tokens were badly thought out and never implemented inside of synapse, and so nobody was using them. We're not aware of any other significant examples.

There are definitely things we could do better there, not least speccing new endpoints, but on the whole our impression is we have done reasonably well at maintaining backwards compatibility with what's implemented.

While we can sympathise with frustration with using Google docs, we’ve found it much more constructive for initial proposals of new features than simply using PRs of OpenAPI changes which tend to focus too much on the API shape rather than the fundamental underlying concepts. We’ve also had complaints that OpenAPI is too time consuming to write when discussing new concepts and APIs that rapidly evolve.

We fully imagine that the way we handle spec contributions will massively evolve over time.

Regardless of how or why the Matrix spec process is currently broken, our major concern with having a fork of the spec is that it becomes a long-lived fork which risks fragmentation of the protocol and community:

• Lack of review of PRs makes errors much more likely - and lack of review by the core team makes divergence even more likely.
• The changes are being merged straight onto master, meaning they can’t be reviewed or easily PRd to the official spec.
• Neither the name nor README of the forked matrix-doc repo indicate that it isn’t the official spec. “Matrix Unofficial” might be a better bet.

If you really want to keep a separate repo we would recommend either:

• Merging changes on to branches other than master
• Or having a new repo that has a clear name and readme describing what is happening, and simply having separate files with the proposed changes.

Either way allows references from issues in the official matrix-doc repo, which would help contributors keep track of proposed updates to the spec.

In its current form the core team will advise not relying on this repo, due to the risk of the documentation being inaccurate or misleading.

There is a feeling that a recurring theme in this whole discussion has been that statements made by the core team have been repeatedly taken and extrapolated by @maxidor to conclusions that were never intended. This has led to frustration due to it consuming a large amount of time which otherwise could be spent working together on Matrix spec proposals and PRs, and it is fair to say that we could have made significant improvements to the spec in the time spent on this so far.

On a final note, it's frustrating for the core team to be accused of not caring about the community. We do. We spend quite a lot of our spare time talking to people, guiding them through PRs, answering questions. We wish we could do more, write more documentation, help people write proposals, but we are currently heavily resource limited and we just don’t have the time to be everywhere and do everything. This is why we are asking folks to help by contributing to the official spec and come together as one community, rather than forking the standard.

[ara4n]

Thanks Erik for collating a more diplomatic and balanced response than my draft answers. Ftr i'd prefer to discuss this in public for maximum transparency and visibility rather than DM (although in practice I'd much rather spend the time reviewing/writing spec PRs to the official repo than continuing the debate further).

[maxidorius]

@ara4n @erikjohnston Thank you for your answer.

One fundamental question has not been addressed in your answer which led to the creation of this repository. I've linked to a reply from Rich to this question and this might fall under the following:

There is a feeling that a recurring theme in this whole discussion has been that statements made by the core team have been repeatedly taken and extrapolated by @maxidor to conclusions that were never intended.

Therefore, I would like to ask this again and ensure you have a fair chance to answer it:
Where can we document, today (not when the funding is resolved), the current behavior of implementations in a specification format, in a way that it can produce an auto-updated page similar to https://matrix.org/speculator/spec/HEAD/, which would include content similar to PR 1039 and 1040?

[Magnap]

A point which I think it's worth making very clear (and which we're going to be updating the README with) is that starting from matrix-doc provides a far better base for documenting the current behavior of the ecosystem than starting from scratch would, since this repository is not meant to supplant the official spec, but is intended to contain spec-style documentation of the sort that would allow someone to write a federating HS without having to reverse-engineer existing implementations.

[ara4n]

I'd suggest keeping the various changes as PRs against a fork of matrix-org/matrix-doc, and feel to point a copy of speculator against it. If you want to maintain an 'unmerged-prs' branch then you are of course welcome to. As your PRs are submitted, reviewed and merged against the official matrix-org/matrix-doc master you'd pull that into your repo's master and minimise the divergence. If your PRs aren’t merged, it’s hopefully for a good reason and the expectation is for contributors to work with the maintainer to improve the contribution in order for it to be aligned with the rest of the standard (rather than forking the whole standard as a means of winning the argument).

The main thing we are objecting here is to merging unreviewed PRs to master of a matrix-org/matrix-doc fork, in a repository whose name & README risks confusion with the official spec, and whose current workflow appears to be set up to diverge from the official spec. So please either merge not-yet-merged-to-matrix-org PRs to a sensibly labelled local branch (with a README which makes it clear that it's an unofficial interpretation of the spec) - or instead work with us to merge them to the official spec. The latter will have the advantage of actually being consistent with the rest of Matrix, and help advance rather than fragment the project.

[ara4n]

@Magnap we are all for filling in the undocumented bits of the spec. please just do it in a format which we can merge into the official spec, and with naming and documentation that makes it clear that it's unofficial and unreviewed exploration so that newbies don't google some undocumented matrix API and assume they've found official reviewed documentation for it.

[maxidorius]

I'd suggest keeping the various changes as PRs against a fork of matrix-org/matrix-doc, and feel to point a copy of speculator against it.

To my understanding, this wouldn't work to address the requirement below:

produce an auto-updated page similar to https://matrix.org/speculator/spec/HEAD/, which would include content similar to PR 1039 and 1040

---

If you want to maintain an 'unmerged-prs' branch then you are of course welcome to

This is, to my knowledge, the only way to do it as we need a single document including all the PRs content that reflect current implementations. And this is what we have done.

As for the rest, I'll be sure to be clear on our still-to-be-written down process:

• matrix-org/matrix-doc is the authoritative content, if it reflects the current implementations. That is up to you to ensure it happens. As long as you create an ecosystem that relies on behaviour that break the spec and only work with your own implementations, this will exists.
• Every PR made against this repo will need to be against our copy of your master branch, and never against our master branch. We will enforce PRs to be first sent against the official repos. Only if they do not get an answer, or involve too much work for the contributor to document current behavior of implementations, we will accept it to our master to be able to generate a single document.
• We always intended to clearly label this generated spec document with big red warning saying "This is not official. This is not the spec of intended behavior. This is documentation in spec format of current behavior".

It seems like we agree?

[ara4n]

If you want to maintain an 'unmerged-prs' branch then you are of course welcome to

This is, to my knowledge, the only way to do it as we need a single document including all the PRs content that reflect current implementations. And this is what we have done.

Except it isn't: you're merging these PRs onto the master branch of your fork of matrix-org/matrix-doc rather than an unmerged-prs branch, and the README for the fork doesn't clarify the unofficial nature of the spec. Agreed that it's close, but it's a subtle but critical difference in terms of avoiding confusing users. Please just rename your master as unmerged-prs or something and change the default branch in github; fix the README; and point a speculator at it with an appropriate explanation of what you're doing to avoid confusion with the official repo.

As long as you create an ecosystem that relies on behaviour that break the spec and only work with your own implementations, this will exists.

...where 'this' is eventually-matrix/matrix-doc? okay, yes: i agree that for as long as matrix-org/matrix-doc is inaccurate or incomplete, folks should be PRing fixes to it to help improve it.

Every PR made against this repo will need to be against our copy of your master branch, and never against our master branch. We will enforce PRs to be first sent against the official repos. Only if they do not get an answer, or involve too much work for the contributor to document current behavior of implementations, we will accept it to our master to be able to generate a single document.

Sounds very reasonable. Please note that the blocking point on matrix-org/matrix-spec-proposals#1039 was not over documenting current behaviour, but the unrelated protocol changes which were also included.

We always intended to clearly label this generated spec document with big red warning saying "This is not official. This is not the spec of intended behavior. This is documentation in spec format of current behavior".

Great... except if you aren't getting review on the PRs you are merging, there is a very high chance that the "documentation in spec format of current behaviour" will be not actually correct... or worse, somehow formalise bugs in the implementations. Hopefully the act of PRing these first against the master branch of matrix-org/matrix-doc will catch these however.

Anyway, yes: it seems like we largely agree. Please let's get on with writing code & spec rather than debating further - and please please consider making the changes we're requesting here to avoid confusion for the wider community.

[maxidorius]

you're merging these PRs onto the master branch of your fork of matrix-org/matrix-doc rather than an unmerged-prs branch

We prefer to call it master currently. We'll take your suggestion into consideration.

the README for the fork doesn't clarify the unofficial nature of the spec

Work in progress, we still don't agree how it should be said or integrated to avoid conflicts down the road. It will be done.

the "documentation in spec format of current behaviour" will be not actually correct... or worse, somehow formalise bugs in the implementations.

Being a bug or not doesn't matter to this repo. This repo is about documenting the current behavior so people can actually build implementations that have a chance to discuss with the ecosystem. It's not possible with the current official spec. I can only invite you to fix your bugs if you don't want to be documented...
And yes, we make sure those are PR'ed against your repo so they have a decent chance to be clarified. But the PR will not be modified if it reflects the behavior.

I please please consider making the changes we're requesting here to avoid confusion for the wider community.

We will and we always wanted to.

[Magnap]

the README for the fork doesn't clarify the unofficial nature of the spec

it's a subtle but critical difference in terms of avoiding confusing users

Making the purposes of our repositories clear is at the top of the project's to-do list, and we're working on further making it clear that we're not official. Eventually Matrix was meant as a pun on Eventual Consistency, not to lay any sort of claim to the Matrix name.

or worse, somehow formalise bugs in the implementations

For this repository, once suitably renamed and README'd, formalizing the buggy behavior of implementations will be considered acceptable to the degree that one needs to be bug-compatible with them in order to interoperate with them in practice

FTR, I would have liked to have this documentation in a format that in the future would allow for more clearly noting which implementations differ how (spec style isn't very suited to that), but considering how compliant the current ecosystem is with the matrix-doc we branched off from, that'd be a lot of work to either port or re-make.

[maxidorius]

@ara4n ignore the part about the PR in my original reply, I totally misread your sentence.

[ara4n]

you're merging these PRs onto the master branch of your fork of matrix-org/matrix-doc rather than an unmerged-prs branch

We prefer to call it master currently. We'll take your suggestion into consideration.

Please either rename the repository or rename the branch. Otherwise anyone talking about "master of matrix-doc" or similar is going to be incredibly confusing, as is anyone trying to merge between the repositories (i.e. us trying to incorporate your changes). Likewise folks on github/google are going to get incredibly confused on stumbling across a repository which seems to be very divergent yet carry the same name.

Please note that the blocking point on matrix-org/matrix-spec-proposals#1039 was not over documenting current behaviour, but the unrelated protocol changes which were also included.

So the core devs have been claiming from the start, even tho I have said it is about current behavior. This also was discussed in a public room prior and said already, even tho I can't find the link anymore. How else could I have said it? Why is the core team simply ignoring some of my comments?

boggles

I am genuinely bewildered by this. Having hunted down the comment (Github has collapsed the comment history because it's so long, so I had to manually hunt in the DOM for the anchor tag - have I mentioned how useless Github PRs are at lengthy proposal debates?), I can only pray that this is some very strange miscommunication. The situation as I see it is this:

• You propose altering the spec to let 3PID mediums be arbitrary strings. Currently the 'reference' implementations (sydent; synapse; matrix-{ios,android,js}-sdk) supports only a fixed set of strings.
• @dbkr points out (having done a sanity check with me IRL) that arbitrary freeform strings are going to cause fragmentation in ID servers and how to handle new identifiers: it's better to restrict the list and define new ones formally as they arrive (or let people experiment within their own namespaces to avoid collisions).
• the conversation then spins off into an interesting spec design conversation about how best to namespace the identifiers. You ask how to unblock the process; @dbkr suggests wording that lets us fix the namespacing problem down the road.
• This then gets derailed again into a discussion about namespacing, and then spec contribution workflow.

...and you've just told me to "ignore the part about the PR", So i'll go ahead and ignore this. Believe it or not we really want to merge your PRs. But that means being open to compromise and pragmatism to some extent during the review process, just like any other FOSS project.

the "documentation in spec format of current behaviour" will be not actually correct... or worse, somehow formalise bugs in the implementations.

Being a bug or not doesn't matter to this repo.

Okay, that's fair enough. But please rename the repo to make the distinction clear.

I please please consider making the changes we're requesting here to avoid confusion for the wider community.

We will and we always wanted to.

Thank you.

@Magnap:

Eventually Matrix was meant as a pun on Eventual Consistency, not to lay any sort of claim to the Matrix name.

Okay, I hope the potential for confusion is clear though, despite the pun. Given the aim here to improve and clarify the spec situation rather than confuse newbies further, I'd beseech you to have a less punny and more selfexplanatory name. Perhaps "Empirical Matrix"? This will also encourage us to improve the official spec so we can go and kill off the Empire ;P Or just "Unofficial Matrix"?

For this repository, once suitably renamed and README'd, formalizing the buggy behavior of implementations will be considered acceptable to the degree that one needs to be bug-compatible with them in order to interoperate with them in practice

Fair enough.

FTR, I would have liked to have this documentation in a format that in the future would allow for more clearly noting which implementations differ how

That would be awesome. In practice, hopefully the differences are sufficiently few and far between today that one can handle them with inlined comment or rationale blocks within the spec (i.e. the written bits of the spec, rather than swagger stuff).

[Magnap]

Speaking for myself only, the current drafting and specification process seems rather "heavy-weight", and for the initial steps of writing the draft and implementing it, it would seem to make sense for them to happen in a more "volatile" and independent environment than you would get PRing them against the official spec or existing implementations, respectively. I hope Eventually Matrix can come play that role, as a staging ground (as Erik described it) for potential new features to develop and stabilize, and to eventually be prepared for PRing against a more official upstream (be it the official spec or a major implementation). Conversely, however, I can see how you might consider that fragmentation, and I would like to stress that it's nothing but a vague hope at this point, and one I would definitely be willing to give up on if a more official such staging area were to exist.

For the near future, however, I expect the main focus of Eventually Matrix will be to collect the results of reverse-engineering (so much easier when you can look at the code, ❤️ open source) the already existing implementations to support the work on making 3rd party HSs (and clients, ISs, and ASs too, if those working on such would like a place to host their results) federate and interoperate.

As for our name, we'll take a second look at how we can make it clear that we're unofficial. Same goes for the name (and README) of this repo. In general, making it more obvious "what goes where" is very much a priority, but is still something we're working on coming to an internal consensus on. But this repo will definitely be modified to better suit its purpose as an unofficial documentation of current behavior! It might take a couple of days, though, as this is a spare time project for all of us. But rest assured that we are not trying to deliberately confuse anyone, and we will clarify whenever necessary.

[ara4n]

Speaking for myself only, the current drafting and specification process seems rather "heavy-weight", and for the initial steps of writing the draft and implementing it, it would seem to make sense for them to happen in a more "volatile" and independent environment than you would get PRing them against the official spec or existing implementations, respectively.

There is some small chance that we may have done quite a lot of drafting & spec of Matrix thus far... which is why we have tried to make the drafting/proposal process as insanely light-weight as possible. At the risk of repeating my original post, all we ask for a draft is a proposal in Google Docs (or something we can c+p into Google Docs to discuss with threaded comments, revisions and track-changes). There isn't even a template.

You then experiment against whatever implementations you like, and then if it's shown to not suck, do the formal PR against the spec. I genuinely can't think of something more volatile & independent? However, if you want to speed up the 'formal spec' phase by experimenting with an alternative copy of the spec, feel free - as long as you PR the proposals against the official spec too, and clearly label the experimental alt copy as such.

[Magnap]

You then experiment against whatever implementations you like

However, if you want to speed up the 'formal spec' phase by experimenting with an alternative copy of the spec, feel free - as long as you PR the proposals against the official spec too, and clearly label the experimental alt copy as such.

Those are the parts I was thinking could happen here, also to allow collaboration on them even while they're still in the experimental phase.

However, strictly speaking, that's a separate point than this issue, and for this issue, it doesn't seem we disagree much on what needs to happen in practice.