Positioning, vision and future direction of the Dat Project

[aschrijver]

(NOTE: This issue has been moved from dat-ecosystem/dat#824)

As a newcomer to Dat and the field of Decentralized Computing I decided to provide my observations, advice and recommendations. I've always found outsider feedback to be refreshing, thus I hope mine is as well.

I've split the original discussion into separate parts:

• part 1: Benefits of slight repositioning, expanding of vision

Explains how Dat can easily reposition as a message-based application platform, and broaden its applicability, strengthen its community and obtain more and easier funding

• part 1a: Analysis of dat community culture and development approach

A swot analysis of the dat community culture and the development approach that results from it

• part 1b: Proposal - Improve dat-awesome page (status: PR'ed)

Existing awesome list is outdated. A complete overhaul would give a better dat ecosystem overview

• part 1c: Proposal - Improving dat project github repository organization

Dat ecosystem is confusing to newcomers. Existing dat module repo's provided by dat project are scattered. Github project organization can be easily improved for more clarity and consistency

• part 1d: Proposal - Provide high-level project tracking of ongoing work and roadmaps

Dat project overall development status, planning and roadmaps are obscure. Providing a simple, high-level means of project tracking will greatly help to engage and involve the community

• part 1e: Proposal - Auto-generated, published and aggregated user and API documentation

Current documentation procedure is inefficient and error-prone, leading to outdated or incorrect documentation. Automation can alleviate most of the current downsides.

---

Messaging proposal

• part 2: Dat: A message-based system on top of raw streams

More details on the technical side, explaining how Dat current design concepts and a proposal for moving towards messaging. The proposal was accepted as useful by @mafintosh @joehand

• part 2a: Design of message-based abstraction layer on top of hypercore

Elaboration on the actual design of the abstraction layer, which takes the project Vert.x as an example

• part 2b: Investigating message abstraction layer implementation options

Evaluates whether to extend the protocol itself, or layer a message abstraction on top of hypercore

---

• part 3: Optimizing traction and exposure

Addresses issues posed by @blahah and @joehand on maintaining stable, dedicated team, good quality output and at the same time avoid negative attention as the project gains more traction

• part 4: Vert.x: A case study of successful project redesign

More details on the vert.x project, which is both a showcase for good project organization and community building, as well as an interesting technology to incorporate with Dat

--
.
Hope my input leads to fruitful discussion and overall benefit!

Arnold Schrijver

[blahah]

@aschrijver the energy and thought that you have put into all of your recent discussion contributions is astounding, so thank you first of all.

I want to manage expectations by highlighting that the projects in these issues are all in similar situations I think - relatively slow but steady community growth, but with a solid theoretical foundation. None of these projects (afaik - I don't speak for them except ScienceFair) have a lot of resources to dedicate to discussion in issues right now.

We definitely appreciate the thought and care, but please don't be offended if people take a long time to respond, or if issues are closed for now to avoid attracting long discussions while the project team don't have time to give them the attention they deserve. With low resources and a technical point to prove, I think most energy is usefully spent in development. That doesn't detract from the importance of the issues you highlight, it just means that it might take a little while for us to match your energy :).

For ScienceFair, I can say that we have thought about a lot of the issues raised but that documenting our ideas has come second to establishing whether they are viable. As soon as the resources we can attract prove the latter, the former will become the priority. I would urge you to hold off on opening any more issues while we have time to digest these things along with the other (invisible on github) things we have going on.

And finally to reiterate, the care you have put into this all is very much appreciated. Thank you :)

[aschrijver]

Thanks @blahah I understand fully, and am be offended at all, don't worry 👍
I ensure you, I know about hard work as well, he he ;)

I find technology concepts and underlying philosophies greatly inspiring and now have some time. I'm in between career options (just had a short sabbatical) and in the opportunity to give my outsider feedback.

What started rather high-level has now become altogether more practical as I managed to convince @mafintosh on the usefulness of having a message layer abstraction.

I just finished a second draft design document on how this can be implemented (see: Investigating options to extend hypercore-protocol for messaging).
Also on ScienceFair and other Dat project I've added altogether more actionable and on-topic comments as of late.

[aschrijver]

PS I might still go on with my social networking plans and become a long-time member of the dat community.

I have some new and radical ideas I would love to try in practice.
But it's also high-risk 'personal-income-wise' so I may still shelve them for later

Good incentives for you to give some extra attention, or at least 'leech' as much of my knowledge, experience and time 😃