Provide high-level project tracking of ongoing work and roadmaps

[aschrijver]

(NOTE This proposal is part 1d of Positioning, vision and future direction of the Dat Project)

Dat Project organization is confusing to newcomers. Gaining good ecosystem overview, figuring out development approach and community size, habits + culture takes about a week.

Too long if you ask me. Ideally a newcomer should be able to make valuable contributions after 1 or 2 days of studying the technology.

What is needed to improve this:

• high-level insight of ongoing projects, project status, epics and roadmaps (but KISS)

Maybe dat project could use one of the promoted project trackers by github, e.g. ZenHub

Benefits

• Transparency. Immediately clear what you are working on, what you have planned
• Engagement. Insight + understanding at a practical level leads to engagement
• Involvement. Much easier to jump in where its needed most, provide accurate feedback
• Planning. Lean, agile product planning makes your life easier

Drawbacks

• Discipline. Requires a bit of effort, attention when incorporated in dev process

---

Github project boards

[Update: tried by dat team, found unwieldy/unsuitable]

I don't know how you do work planning currently, besides the ad-hoc chatter on freenode #dat (which I consider unsuitable for this purpose).

But you could consider switching on the project board feature that is in all of your repo's.

Github offers 2 kinds of project boards:

1. repository project boards are scoped to issues, pull requests, and notes within a single repository.
2. organization-wide project boards can contain issues and pull requests from any repository that belongs to an organization.

Or, translated to sensible Dat Project terminology:

1. module development tracking
2. feature and roadmap tracking

The boards are incredibly simple to use (a scaled-down Kanban even). Switching them on is incredibly easy.

---

Next part: Auto-generated, published and aggregated user and API documentation

[joehand]

Dat Project organization is confusing to newcomers. Gaining good ecosystem overview, figuring out development approach and community size, habits + culture takes about a week.

Ya, I'd agree this is too hard right now. One of those communication bits we need to work on.

Right now, we mostly track our organization-wide work via our irc standup bot. This gets posted to a private channel. We can make it to also post on #dat IRC channel (I added that functionality last week) and/or posted on a public website.

All of our module tracking happens in milestones. We are more active in using those if there are >1 person working together on a module concurrently. But sometimes we do a bad job when it's just one of us.

Re: project boards, we tried to use them a few times (both via waffle.io and github) and found them lacking and difficult to manage. Our project boards got out of date more often than not, causing more confusion than worthwhile. Part of this was because we focused on milestones within a repo and also because of development happening across many repos and users (some of which couldn't go in the project boards).

I'd agree some method of communicating this would benefit transparency and engagement. We'd be open to ideas on how to track our large strategic planning, probably closer to the science fair strategy. On the smaller timeframes, between our standup and within-repo milestone tracking, we have a good handle of what is happening but can do better at surfacing those publicly.

A lot of our community involvement happens on specific modules which also reflects how we work. A community member can jump in and contribute to a part of the ecosystem without needing to understand how everything works together. This is why we see much more contributions on our dependencies than our user-facing apps. While this is not ideal, we'd prefer folks can jump in to any point without needing to understand the whole. But we can improve communication for those that want to take the opposite approach.

[aschrijver]

Nice idea the IRC standup bot, and its eating your own dogfood, so to say. Publishing this feed would already be a good help. I would not use #dat as it is mostly for chattering and older info would become lost, unfindable in the long discussion thread.

Besides publishing to website, you could just publish to a dedicated repo (dev), either as files, issues, or wiki pages.

With regards to project boards, maybe you should forget tracking dev planning there, but instead focus on the feature + roadmap part (which is most helpful for newcomers to find their way quickly).

You could have one repository, say dev to track the aggregated state of development, planning, roadmap across all core dat ecosystem modules. And only use a project board here, for the high-level stuff (thus needs less frequent updates). Record decisions, both open and closed that came from standup, or more strategic planning
Then give, if it all works smoothly, give it its own gh-pages e.g. dev.datproject.org

I reviewed the science fair strategy and gave some tips. I think it you should have such a document for Dat Project, and maybe have it in the dev project as well. Also Code of conduct, Coding guidelines, Contribution guidelines, etc. would be useful and potentially save you much time.

Don't know about a proper toolchain that fits Dat. I have much experience with Atlassian, and know you could cope with Confluence and Jira, on the condition it is properly configured (imho, bad configs are mostly reason why people dislike Atlassian). As non-profit you can use it for free.

Besides this whole discussion I found CI/CD, continuous improvement strategy to be very effective. Spend time once, automate some task, save more time later. Automate anything and everything.
I am running a personal CD pipeline of confluence + drawio, jira, bitbucket, gocd + docker for a number of years. Now I'm looking into simplifying to just Confluence + Gitlab + Rancher. PS Gitlab allows groupings of projects

You should also investigate what docker can mean to you, e.g. automatically testing downstream projects for compatibility / breaking changes, testing in different environments, setting up large p2p test networks, etc.
(off-topic: I've found mostly unit tests, not much focus on integration, performance, security, guerilla, chaos monkey, compatibility, etc. testing, time-consuming but increasingly important as ecosystem expands)

Back to original topic:

• If you can highlight some high-level requirements in a wiki page, I'd be willing to help you turn it into a plan, do some research. We go for highest benefits with smallest efforts and a list of follow-ups that community members can pick up on

[aschrijver]

PS With regards to forums, this might be a nice idea: https://github.com/FriendCode/gitrap (outdated, but a nice idea)

[aschrijver]

@joehand I updated the original issue text based on our previous discussion.

PS Stuff like ZenHub looks very interesting and can be used for free with open source repo's