Implement The Matrix Protocol #245
Comments
|
This sounds subjective, and it almost is, but matrix DOES NOT ACCOMPLISH MINIMALISM, which has been a major attraction of IRC ever since other chat protocols popped up. It also tends towards inexorably high data usage compared to an equivalent IRC use load. IRC 2.8 is like a Prius. Matrix is like a souped up Cummins-powered Dodge RAM 3500. Matrix does not share many spiritual similarities with IRCv2.8, and is barely comparable to "IRCv3", or IRC 2.8+pdpc. If WiZ wishes to weigh in, he is fully and totally within his right to, and his decision as to whether Matrix is IRC Three will not be challenged by me. I cannot attest to the mainstream usability of IRC 2.8 into the future, but it remains the easiest useful protocol to implement. The ease of implementation de novo, minimalism, ability in most implementations to connect without prior identification (with the option to, as on DALnet which pioneered Services in the first instance, Freenode which brought SASL to the front, and uncountable other IRC networks, identify oneself after or sometimes during connection), and ability to choose not to be affiliated with a network in an intuitive manner (just don't set up any C: or N: lines!) is a good case for, at minimum, not repealing IRC 2.8. This is quite subjective and nichey, but an IRCd or simpler yet a mere IRC bot could be written for a microcontroller and it wouldn't be too much of a pfaff to get it to work (minimum code length for a full-stack implementation of a client is far shorter for IRC, which merely needs to do raw sockets and text parsing, whereas Matrix requires a complex markup language) as an IoT control panel, perhaps even the way for a web frontend to forward its requests to the µcontroller handling your lights (launch a one shot IRC client into the background, and send the request, pushing the response back by a means of your choice). Matrix could be a viable alternative for Joe Average (and indeed, is now) but is still completely unusable for its intended use case in many geographic locations that are otherwise considered modern (Canada, for one - mobile use of Matrix is largely impossible due to the untenably low data limits of even the costliest mobile data plans). No constructive solution to these problems presently exists that doesn't involve ditching most of the advantages of Matrix. |
Matrix is actually not hard to implement once you operate with certain assumptions. These assumptions are extremely realistic in this era: that of the very basic web ecosystem. HTTP and JSON are quite possibly the most widely supported methods of interchange in the history of computing. Most languages, frameworks, whatever, almost surely have these not just as libraries, but built in. Furthermore, if you analyze them closely the overhead for them alone is very small, and still laughably small for any device manufactured well over a decade ago. I have already written a Matrix client implementation as an SPA in a browser using nothing but HTML/CSS/JS, and I did this while I was sick in bed, up and running within a day. A child could implement this protocol -- and that is the point. That is where mIRC came from and that's why the next teenager in a basement will be able to succeed. Matrix is not heavy. Indeed the naive implementation will suffer from having to deal with large state transfers; for example the #matrix:matrix.org members list: over 10,000 state events have to be transferred for that channel the same way the NAMLIST in IRC2 has to be transferred. Except it doesn't. Matrix has a fully configurable system of filters and sync options so that your client only has to transfer the parts of the state that it needs based on what you want out of your UX.
One of the reasons I found Matrix easy to implement stems from the fact that after a trivial guest registration, I was able to make a /sync request and be up and running with all of the information I needed from scratch on every page load. This is minimalism to me. Remember, IRC2 requires a NICK command, a USER command, then you get the MOTD and some other messages. You have to run commands to JOIN all of your channels (slowly, otherwise you are shunned with a whip out in the woodshed) -- and you learn nothing about the past sans the current membership list. With matrix, you actually do less work yet your client learns everything about what you really want to know: the channels you're subscribed to, the messages you missed since your last login, etc. That's the minimalism right there.
Again a well formed client will make intelligent decisions about what it wants to see and what it does not. Your client should set a server-side filter to only receive messages for the channel that you're currently focused on. What about being pinged from another channel? Don't worry, they thought of that. IRC2 can't do any of this, so what's the solution? Just limit the user to CHANMAX:100 channels. And when 100 channels isn't enough? Get on your knees and talk to an ope.... oh ffs.
Old IRC first requires a sequence of negotiations for NICK and USER -- and these aren't really meaningful identification and so you have to shellac all this other stuff on like SASL and NickServ. Matrix rectifies this by having a single uniform system as you'd hope for. If you want to be identification free, there is a Guest Registration which auto-generates an identity to get you up and running in a single request. This is less work than IRC2. If you want even less work, many requests require no authentication and you could theoretically read the entire history of a channel, and even actively watch it without having any identity or presence in it whatsoever. IRC2 can't even hold a candle to this.
Indeed, if I had to show up to a class tomorrow with a working FPGA using VHDL I'd probably crap myself having to get up and running with HTTP/JSON/Matrix in contrast to IRC. But I don't... And though I'm not deep into IoT devices, I think they're running full mainstream operating systems with web browsers now like anything else. That statement in itself is an argument for matrix (hint: web browsers!). Lastly, let me just assert that it's fair to say Matrix supersets IRC. Not with a syntactic backwards-compatibility of course, but with the basic features and feel you'd expect out of the next version of the IRC protocol. This is evident in the ease of creating a Matrix to IRC bridge. Such a bridge already exists, functioning between matrix.org and freenode.net. |
|
Personally I think this might be out of scope for Charybdis as I don't remember seeing any IRCd supporting multiple protocols, but I don't have hard opinions to either side and I just happened to see this issue in my email. @jevolk You might be interested in matrix-org/matrix-appservice-irc#329 which is implementing TS6 in matrix-appservice-irc letting it talk with Charybdis.
Also many other IRC networks as seen at https://github.com/matrix-org/matrix-appservice-irc/wiki/Bridged-IRC-networks . |
But that's really the point of this discussion. I'm saying it's self-defeating to consider this another protocol. This is the protocol. There won't be anything better out of #ircv3:freenode.net. Please excuse my matrix federation addressing in that last statement, because I couldn't use IRCv3 federation protocol addressing, because it doesn't exist. This is just how the cookie crumbles. If you're still hoping for something else that is more syntactically similar to IRC2 with a bunch of features that matrix already has deployed, let's use this thread to convince you why that isn't necessary. |
|
If you are sure that this is what you want then feel free to go ahead and switch protocols. If there are any Charybdis users who wish to remain with the IRC protocol I will be happy to help them migrate to a different server implementation with full IRCv3 support like InspIRCd. Good luck with your future endeavours! 😊👍🏻 |
|
@SaberUK There are no plans to drop support for the IRC protocol versions up to the current freenode-compatibility level. These messages will just be translated to Matrix. Seeing as how Matrix still supersets all of the drafts by #IRCv3:freenode.net, I think it would be possible to implement something similar to the Matrix bridge but for the complete #IRCv3:freenode.net suite of specifications. All of this can be done within the server. |
I don't really have any opinion on whether charybdis should be relaunched as a matrix server or not, but I had to reread this paragraph several times to determine if it was a joke or not. I mean, given the current state of IoT security, I wouldn't be terribly surprised to find an IRCd running on my refrigerator, but still... seems to me like both matrix and IRC are the wrong options for IoT, you would probably want to use something like MQTT instead, no? |
|
I personally would actually run an IRCd on my fridge if it meant I could request that the ice machine start up, or check how much ice is in it.
|
Why IRCd? The fridge is where you'd run a client which connects to a central IRCd 'command hub' with other devices as well. What happens when the power goes out for a few hours? You'd want the fridge to turn on and download and process all of the messages it missed. You can't do that with IRC. The app which sends the commands would have to know the fridge isn't online, wait for it to connect, and then re-transmit the queue of commands. With matrix all of that is uniformly baked into the core of the protocol. |
Well I don't want to build Cray into my fridge, so no. Additionally, if my fridge goes out, I don't want it to process commands that should have been executed hours ago, because that could be Very Bad (like if I commanded it to make ice and then it booted up, the power consumption could pop a breaker if we were running high % of max amps already) |
Here's a reason for why I find an extensible interchange format (i.e JSON, or anything similar) to be an advantage. Normally for your case I'd expect to hear myself barking back "that's a problem for your application logic, not the communication system. Have the device know when to ignore stale commands." But with a system like Matrix, we can start to blur the lines by encapsulating your application logic within the messages themselves. For example, when the fridge reads any message that includes some field like But that's really scratching the surface. We can go a lot further than that. We can start to encapsulate things like entire javascripts in fields in messages, and define both the logic and state for the fridge dynamically. The possibilities are infinite here... Corollary edit: I want to point out that there's an advantage to having the whole system use the same extensible format like JSON. In other words, the same state format for low-level aspects of Matrix and the state for your application like a fridge. This allows IRCd to become more intelligent than a simple message passing device. It can conduct deeper inspection into your messages. IRCd can be programmed with modules and even dynamically with javascripts to do application-specific things for you, like dropping messages with |
|
Yes the possibilities are infinite, and the processing power required is too. |
|
While the IoT discussion is amusing, the merits of Matrix protocol are only relevant if Charybdis aims to be a multiprotocol chat server in the first place (and if its existing 2.8 core is suitable for such advanced features as you've described). I believe the needs of Matrix users would be better served by a modern from-scratch implementation (e.g. the Rust-based matrix-ircd) or by reusing the guts of, say, an XMPP daemon. (Meanwhile, if you're "worried" about the bleak future and wasted efforts of IRCv3 – not your problem. Just close your eyes, take the blue pill, and pretend we're retrocomputing hobbyists or something.) |
I opened this thread with the subtext of (and several followup arguments for) Matrix not being another protocol and Charybdis not intending to be a multi-protocol chat server. This isn't the Pidgin IRCd; this issue isn't about supporting Oscar, Gadu-Gadu and GTalk... I made it clear that Matrix achieves the goals the next iteration of the IRC protocol wants and needs. That is evident by the IRCv3-WG attempting some of the same ideas concurrently, like Chat History -- but only some. @grawity The truth here is that I don't care about your politics. I just write the code from the spec you hand down. The fact here is that the IRCv3-wg simply hasn't handed me nearly as much to work with as the Matrix-wg has. Barely a fraction of it, if that. I opened this thread to make that clear to other developers. The facts are objective here, and the evidence is in plain view for everyone to see. This isn't even my opinion at this point. Matrix now has widespread deployment, real-world testing, and it iterates quickly. Any counter-claims by IRCv3 compare fractionally to those statements at best. If those facts were the inverse and your spec looked anything close to this, that would be the winner fair and square. But it's just not.
It may be, it may not be. I don't know yet if we're going to limit this to C11 or bring out C++14 -- and frankly we can have translation units of both varieties sharing headers with compatible interfaces. That's something we can discuss as an implementation team and I'm open to anything. We can't get to that discussion if you close this thread.
Your opinion is noted, and may very well be absolutely true, but it doesn't address any of the arguments I've proffered. Nor does it address why it doesn't address any of the arguments, and it's not a reason to close this thread.
This most of all provides me with every reason to re-open this discussion. You're developing something behind closed doors, reassuring me about its promise, and then coming up short with anything real to work with. Are you developing a protocol for IRC or the united states congress? |
|
Matrix is Matrix, Charybdis is an IRC daemon. I believe your argument is to steer Charybdis away from IRC and implement an entirely new protocol. What's the point in reusing Charybdis at that point? You're pretty much rewriting a toaster to become a television. You've mentioned many times that you do not like the current code base (ref: #198), so why do you feel the need to gut it to the point where you'll pretty much be the only one working on it? We're hobbyists that enjoy dabbling with IRC in our free time, we're not looking to maintain a new spec for something most of us don't even use, in a language some aren't comfortable with, and libraries most of us dislike. My question isn't "Why do you think we need Matrix?", it's "Why do you want to take a legacy code base, completely rewrite it, and make it do something entirely different?". It seems like it'd be less effort to start from scratch... |
|
@staticfox Okay, so the year is 1995 and you see Charybdis as a 1980's BBS daemon and Matrix as a new hip web bulletin board. Is that a fair analogy? I'm really not mocking here. I don't have a real problem with that view on its own. What isn't right, and isn't necessary, is trying to hinder forward iteration. How many BBS daemons can you name off the top of your head? How relevant are any of them today? I'm glad this is your hobby. There's real value in maintaining the hobbyist snapshot branch. But this project can either move forward or it can become irrelevant too. This is why we need Matrix. Matrix is the next iteration of IRC.
As an anecdote, so far 80% of people I've proselytized matrix to have switched and use freenode from matrix if necessary. The last 20% actually switched too but still uses an old IRC client and a riot instance for history when they're disconnected. @staticfox So now I have to ask, are we debating what the next iteration is or are we debating whether Charybdis iterates at all?
@staticfox The error you're making here is simple: you're looking at the name. You see that it's called "Matrix" and not "IRCvX" and you're defaulting to a conservative position. Look at its substance. Look at what it has to offer under the hood in contrast to IRCv3. If you're not convinced, look at the ease of creating an IRC bridge and how the two translate to each other. It's the tao forward. |
|
@jevolk The error you're making is thinking that Matrix and IRCv3 as having even barely similar enough interest bases. IRCv3's interest base is IRCers. Matrix's interest base is Joe Blow. Also, you'll have to revise your 20% figure down a tad. I tried Matrix, it failed me. Perhaps Matrix could be posited as one of many implementations of an "IRC-NG" or "IRCv4" concept. |
I open up the matrix.org room directory and I see my favorite open source projects.
These are all channels independent from freenode and have userbases rivaling or surpassing the legacy counterpart. Come on @janicez, I'm not exactly doing dissertation level research to make these posts but have you even spent a minute to check out what you're saying or are you content with spurting nonsense? I expect a lot better from someone as intelligent and articulate as yourself... |
|
@jevolk You know, I hate to say this. I really really hate to say this. But I completely see where you're coming from. I understand things evolve and it's matter of time before something either steps up or becomes irrelevant. One thing I still stand by that you didn't answer, why specifically do you want to use Charybdis for Matrix when you've openly stated you do not like the code base at all? |
Because if Matrix really is the next iteration of IRC then it's our job to do this. Whether or not 10% or 100% of the code-base needs to be rewritten is an implementation detail. This is the task for Charybdis now. If we don't do this, then IRC and Matrix have separate and independent futures. But they don't. @staticfox The future is now. Are you ready to seize this moment? |
|
What I will say is that the original people who started charybdis don't work on it anymore. In my opinion, I think the value of charybdis reimplemented as a Matrix server is low -- a Matrix server with the same approach as used for charybdis is worth doing, but the brand is meaningless there. Additionally, I would ask that anybody still hacking on charybdis IRCd to rename the project to something else as the hacking appears to be based on the charybdis 3.5 branch instead of the 4 branch, which in my view (as the person who started the project) was supposed to be the future. |
@janicez So I tried riot mobile on android... it is terribad. It does eat up a ton of battery. When it doesn't crash, it doesn't seem to consume very much data- but then it crashes and seems to resync all kinds of data again.
From what I know about the matrix protocol and the pitfalls of how it ends up being implemented naively, there are a lot of solutions here. I haven't checked riot.im mobile app source but I'll make some assumptions for constructive fodder: First, I can't imagine riot is making ubiquitous use of filters. In fact, if I were to design another UI and surely for a mobile UI, I'd probably build it around filters themselves rather than tacking them on as optimizations. The protocol spec may be misleading here. They discourage the use of one-off filters in favor of pre-registering them. This probably leads to implementations not taking them seriously enough. Second, I don't think riot mobile is making adequate use of cached data, especially content addressable data like matrix events. That's why matrix events are really cool by the way, they are content addressable. Nothing in IRC is content addressable. What's cool about this is that better filters can be written to send nothing except I don't think these things burden the developer too much if they knew to consider them beforehand. Matrix is such a new protocol and the spec doesn't really comment on these kinds of things yet. |
|
I'm not really sold on Matrix, but I would like to see charybdis commit to federation in some way. Such a commitment would be enough to make me work on it again, despite the threats from certain IRC networks related to FOSS. |
|
Matrix dev here (sorry for butting in, but there's some interesting stuff here): Yup, today's Matrix implementations are unashamedly heavy. This comes in various flavours:
HOWEVER, the project is still in beta, and the intention is categorically to fix all of this. Some clients (e.g. nheko) are already using filters much more intelligently; Dendrite is roughly 2 orders of magnitude faster than Synapse; and there are various better-than-HTTP+JSON transports in development (e.g. https://github.com/matrix-org/matrix-doc/blob/master/drafts/websockets.rst. and we'd love to see COAP+CBOR or capnproto based impls). Now, I get that this is ideologically diametrically opposed to IRC, where the protocol started from the most minimal and efficient starting point possible and then stuff got strapped on; instead we've started from a non-optimised bloated solution and then moved to optimise it as time goes on. It's possible we oversteered and started too bloated, but the end result will hopefully be the same - we'll converge on something elegant (whilst still having compatibility with the keep-it-simple-stupid HTTP+JSON baseline protocol). Hope this clarifies the rationale a bit. Whether it's appropriate to implement it in charybdis isn't my call, but the idea of a C/C++ homeserver would be amazing. |
|
oh, and @janicez:
Despite Matrix's bloatiness, it does work well enough for IOT things (assuming they are not massively bandwidth constrained) - e.g. https://github.com/matt-williams/matrix-esp8266/blob/master/matrix-esp8266.ino is a simple matrix client for an ESP8266 that ended up shoving messages on an LCD display. I'm not sure I'd call JSON a complex markup language :) And there are many examples of folks using Matrix to control their lights already - e.g. https://fort.kickass.systems/git/rrix/lightrix ;) |
|
I am wholly amazed that this hasn't been closed as an entirely unproductive use of the Charybdis project's time. |
|
@janicez https://github.com/jevolk/charybdis PR coming soon ofc. |
|
I don't really have any opinion on whether charybdis should be relaunched as a matrix server or not, but I had to reread this paragraph several times to determine if it was a joke or not. I mean, given the current state of IoT security, I wouldn't be terribly surprised to find an IRCd running on my refrigerator, but still... seems to me like both matrix and IRC are the wrong options for IoT, you would probably want to use something like MQTT instead, no? |
@davidfrn Hey, we're over at https://github.com/matrix-construct/construct. It turns out that @janicez was right and continued poor management has lead Matrix further astray as a candidate to legitimately replace IRC and federate the networks. It may still grow a bit but it feels like the poor quality of the centralized control Matrix.org has exerted is a crutch that will always keep it niche. At this point I just hope Construct can serve as a launchpad for better approaches to federation at scale. I've considered looking into some of @kaniini 's good work on pleroma but the approach I see there signals to me that this entire "chatroom" paradigm itself is the problem to be solved. Maybe it is. Nevertheless, at a higher level, the application I really want any of these protocols to service is just a real-time collaborative development environment in a decentralized setting. I still have a dream this can be achieved, but people have to cooperate for that; ironically. |
|
In case there is interest, https://github.com/privacytoolsIO/privacytools.io/pull/562#issuecomment-457878353 begins strings of comments from someone who used to run one of the biggest public Matrix homeservers at that time and there is also a comment from Matthew who also commented here. I don't know if there are any news on better homeserver implementations or is Synapse with these issues the only choice or would these issues start following Charybdis if the Matrix protocol was implemented. |
That touches on what fascinates me about this entire ecosystem. Every single individual in it wants something different out of it. Every party brings different needs to the table; often enough those are exclusive to others' at the same table. Some want privacy, others want functionality, neither knows the extent to which they really need them; both try to engineer a panacea, and then convince the other to cede when they cannot. Still, others criticize the locus of any approach: is it too technical with a bad UX? Is it too flashy without any attention to security and scalability? Is it controlled by one tyrannical party? Is it too schismatic from warring factions? Finally, there are those who want to capitalize on this orgy by crafting cults of grandiose promise and popularity. Their bread and circuses satisfy everyone's exclusive desires all at the same time, of course, until the bait is switched, all while the oxygen is sucked away from something else that would have been real. So these are all matters for the sociologists. All I want is to do is engineer a uniform communication infrastructure. A simple and reliable platform under this circus. |
|
@jevolk Hey, well I’m truly interested in working on aspIRCd with you. aspIRCd is based on Charybdis with some further extended modules that the UnrealIRCd uses like promoting the user to Owner upon a Channel Registration , plus , we have a variety of encryptions and whilst having some of the coding from ChatIRCd. I would love to work on it and further work with you on aspIRCd. I would request you to meet me up as soon as possible on the aspIRCd development server - irc.aspircd.com Port 6667(plaintext)/6697(SSL) #aspIRCd there! I’d like to meet you as soon as possible there for some future development reasons. |
Consider me spoiled at this point, but I just don't see a future in twiddling a few knobs and then pulling off the cover shouting "here it is folks! IRC is back!" We had these debates ad nauseam years ago. The conclusion isn't even in any of our hands to make. Everything else that exists: slack, matrix, whatever, is there because IRC has totally reached its limitations in many ways: technical, social, political... What needs to be done is certainly not compatible with the current ecosystem of software- then it will be said that whatever is done is not IRC anymore. What is the point? @davidfrn Give another of these services a test-drive, just for a little bit -- just once. I'm sure you'll find some killer-feature in one that IRC doesn't have. You'll probably think about how you can import that feature back to IRC; perhaps even write a PoC with aspircd. Clients won't support it. So maybe you'll go around and patch all of the clients. Other servers won't support it. So maybe you'll go around and patch all of the servers. The networks won't run the patched version. So maybe you'll start a new network- well it turns out that the ecosystem has so much attrition that it won't have users. Despite sounding cynical, there is hope. There is still a vacuum to be filled for a FOSS successor to IRC. The vacuum is there. But it won't be filled by full-stack-integrated comprehensive-solution X that supports Y and depends on Z. Matrix came close because it started with a powerful abstraction. It wanted to satisfy everyone's desires all at the same time to gain popularity, and it stumbled on a certain extensibility that resonated with developers. People saw possibilities for where they could take it with their creations. They felt it gave them choice. That was an illusion. I now think the way out of this is with a platform of some kind. It has to be abstract enough to satisfy competing needs; small and simple enough to be universally implementable; extensible enough to carry diverse features; yet remain uniform with a continuous surface, at least at its lowest level, which resists fragmentation from factions in disagreement. That sounds like some kind of pipe-dream, unicorn protocol -- but it's not. The internet itself is an instance of such a platform. Any other approach is a roll of the popularity dice, and just as fleeting. |
|
@jevolk yeah, but the main thing I requested to you was to come to the aspIRCd’s Official Network , because , it is a bit too unconvinient to talk here again and again. I’d just for the last time request to you to come there and be there , because that is the fastest way we can sprall are views together. Thank you. |
|
IRC is still alive , mind that! I’d just require you to be at my Development Server As soon as possible. |
|
@kaniini This issue has gone off the rails and been used by Vibhore Changy of Hyphovy Networks to advance Hyphovy's commercial agenda (to the extent that they have one). Requesting closure. |
The matrix protocol has emerged as the 3rd iteration of and future direction of the IRC protocol. This protocol was developed independently of the freenode-based IRCv3 working group, but encompasses virtually all of the design goals for what that group was trying to accomplish -- and then some. It already has a plethora of implementations, and the size of its ecosystem and userbase is rivaling and beginning to pass that of IRC entirely. It's beyond clear at this point that this is the way the future is going to look-- in fact that statement is already old news. It's here; it's the present reality; and it's currently being stretched to its limit of scale -- and this is where we come in. This protocol is so easy to implement in the higher level languages and that is where the ecosystem has tended. This protocol now needs a C/C++ implementation.
I'm sure the preceding statements will be immaculately controversial, perhaps religiously so. Bring it on, if you disagree.
The text was updated successfully, but these errors were encountered: