Project Management Process discussion #125
Comments
I don't think this is viable, given that GH Milestones doesn't work across repos. Again, please distill out the model from the mechanics. We can have things called "milestones" that are logical units of work, and implement them as any of {GH Milestones, GH Issues, GH Labels, something else}. The important thing is to get the abstraction right, and working with all the other things (many repos, many issue boards, etc), maximizing clarity + utility and minimizing work overhead. Get the Model right first. Then implement it with whatever Mechanics need to happen to satisfy constraints (eg many repos) and minimize overhead work.
People work in different ways. This reminds me of the blog pipeline comments. It will be useful for different people to have different views, so they need to be able to use labels + waffle boards to get the view that fits their project and what they want. That doesn't have to detract at all from coming up with a unified "what's being worked on pipeline", which is what @haadcode's concerns are about. I'm also very concerned about this-- we should have a global project-wide protocol for how to see "what's being worked on", and having the same labels + same board style will enable https://waffle.io/ipfs/ipfs to show it across all repos. This just means we need to implement both. What we need is to agree upon what we want to be common across all repos. If we want that to be a kanban style label set, then we can do that. But that doesn't mean @diasdavid can't implement a column based view + backlog to fit his needs too. it just means a different Waffle board.[0] We can have our cake and eat it too. Be wary of overheads. [0] I've already checked with Waffle.io and we can do this by creating another repo-- basically, one "root repo per waffle board with one set of issues". If this is not clear, let me know and i can write a little guide or do a video of what I mean here. again, this is mechanics, not model. |
|
thank you for providing feedback @haadcode :) Getting everything tracked, here is the bits from the sidechannel David:
Haad: You are able to find a full breakdown of the ROADMAP https://github.com/ipfs/js-ipfs/blob/a8c5db3152f8131f22573fe204a70411bc5ac4f2/ROADMAP.md The reason why I used labels instead of milestones was exactly because waffle didn't support using the feature 'milestone' across repos, labels are just fine. I believe I understand the reasonings behind kanban and why the model was built this way, but as I mentioned a couple weeks and I reiterate my point, kanban, scrum, agile and other methodologies were designed for close door development, not open source. We, as an open source project, need to be very open and explicit of what is our timeline, explain where our focus is being put and where contributions can add most value (I get this question a lot), that is why I went in the route of doing a column for each milestone, so that we have a clear way to show intent and priority. Note that we still have the |
👍 👍
Is it possible to have a repo, that imports all the other structure/tagging/organisation from the child repos that we waffle'd? |
I'm not sure what you mean? can you expand on this question? FWIW, AFAIK (subject to error) Waffle's model is (ignore GH for now):
Waffle's mechanics (as implemented on GH) are:
This is a very sound model. It could be improved by decoupling waffle boards from a main repo, or allowing a waffle board to be created independent of a main repo (turning a Board into a set of repos, not a tuple "(main repo, other repos)"), but it simplifies their implementation a lot to do it this way. |
|
@diasdavid https://waffle.io/ipfs/ipfs pulls in from many repos. does that answer your question? |
|
@jbenet from what you said above, I understood that there was going to be a feature were we could aggregate and reuse the filtering, tagging and organisation of other board (i.e. https://waffle.io/ipfs/js-ipfs), so that we had the "mother of all waffles" that joined all the waffles together. It seems it doesn't Note: https://waffle.io/ipfs/js-ipfs gathers all the js-ipfs repos, in the same way that https://waffle.io/libp2p/libp2p does it (note that in this one, I haven't gone through organising them all yet) |
|
There's a lot here to reply to and a lot of confusion around what the project management process is. I'm glad @jbenet you opened your definition of the model and mechanics because it's very different from what I mean by them in the context of (management) processes. So as @jbenet points out, it'd be useful to start from the top (the model) and work our way down. I left Lisbon with the impression that we had agreed on this, but my mistake, we need to have this conversation now. I created a presentation to nail the model first. See/hear the attached files and let me know your thoughts and which parts you can agree with. Let's then move to the mechanics once we have an agreement on the model. Slides: And narration: As for previous discussion:
There are several reasons why labels/issues don't work as well GH Milestones: milestones give you a logical grouping of information against a deadline (structure, relationship and timing of goals and tasks), milestones work as a signal (== you see a milestone name attached as a label) both in GH and Waffle, you can't create sub-issues in GH for grouping sub-goals into bigger goals and milestones give you progress data (how many tasks out of how many have been finished). It is true that milestones don't work across repos. But before we disregard its usage because of overhead concerns, let's look at the whole first. As an aside, perhaps we can get "milestones across repos" feature into GH or Waffle by asking them?
You're talking about boards as a "view to (specific) data" whereas that is not their (only) function. They are a way to track where things are at, not for sorting issues and displaying them in customizable views. What I'm saying is, this tool (the concept of a pipeline and Waffle) is not meant to be used in that way. There's, however, a deeper point to this: we need to decide how we want to manage our process across the organization. If each individual project lead manages their project differently, we'll have a mess. If we can agree on a common way (the model), we'll be able to reach the goals of our overall project management as well as fix couple of big problems we have currently (mainly information duplication to a point where it's counter-productive). If we have an agreement on the different steps in the process, the terminology and how we generate and display the information across the process (from a high "quarterly" level to a detailed "daily" level), then we can avoid overhead, improve on the process and generally look at it holistically. If we make it customizable to everyone, then we allow ourselves to the overhead and confusion produced by 1001 ways of doing it.
This is what I mean. It means exactly that we need to agree upon what the process model is and implement mechanics to reflect that. Not implement mechanics and see if they fit. The tool for the model here is "a signal board (kanban)" and it has certain properties (in this case "a pipeline is: the steps needed to deliver value") and if you change the properties (in this case to: "a view to the backlog"), you break the model. edited:
As far as I can tell, there's nothing intrinsically unique about OS software development that rules out using the aforementioned processes. The need for information in closed-doors development is definitely as high, if not more, as for OS. However, again, we need to separate the tools from the model and think of it as a whole: what information we produce (or better, what information we produce only once), what's the way to capture that information and customize views into that data. What I'm trying to say is, I agree 100% with what you say re. the importance of that information, but that information (time-dimension) should be in different form, not on the board. Let's keep the discussion going! <3 |
|
ccing @whyrusleeping and @em-ly, please add other who should be part of this discussion. |
|
@haadcode the kanban style boards you propose are very appealing to me. I'm pretty open to trying something (no strong opinions on the exact way we should do project management) but my only strong requirement is that we have full buy-in from the entire team. |
|
@haadcode thank you very much for preparing that. it's always helpful to visualize. I unfortunately cannot hear your narration because its not all on the network right now (did you pin it with pinbot?). I look forward to hearing the narration. In the mean time, i will respond below to your post above. @haadcode the way you use "model" and "mechanics" is very different from what i've been saying the last weeks. Could you use different words for yours? because otherwise it will be very confusing. Here's what I mean by model and mechanics:
Think:
In our case:
Does this make sense? I can try explaining another way.
These are all mechanics (by my definition). They will have many kinds tradeoffs, and even hard constraints in some cases. They can be figured out, but strictly AFTER figuring out the model. The model should be sound first, and should be able to be expressed clearly and succinctly without ever invoking any github feature. It should be implementable using paper if need be.
Not trivial. i've asked groups like this before for stuff and it takes months or years. Github is a no go completely, this has been requested for a long time. (Many years)
This tool can be used as a Kanban pipleine, or other pipelines. Saying "this tool (the concept of a pipeline and Waffle) is not meant to be used in that way." is not helpful. It's a tool, and it can be used in many different ways, to benefit the user however the user wants to benefit herself. This is why their labels are customizable, they let you pick whatever you want. Yes, Kanban boards represent it in one way, and it has a specific set of columns, but that's just one pipeline, it's not the only pipeline ever. When i use the word "pipeline" i do not mean "the kanban process". I mean "a process with stages".
This is the disagreement we had about the blog pipeline. I chose to use Waffle to display the stages of a blog post moving through a set of stages, because it is very useful to visualize how many blog posts are in flight, what part of the process they're in, estimate rate of movement through the pipeline, how close we are to having a set of posts ready to ship, etc. It was not a kanban pipeline. I'ts a different pipeline, with different stages, with a different purpose. The kanban pipeline can STILL be applied to the blog repo and its issues, because the kanban pipeline cuts into a different set of things to look at, more focused on immediate work to be done, what's in progress, what's blocked, what's done. Waffle boards are a way to bind the issues of many github repos into a pipeline, where stages are labels. That's it. We can use Waffle to implement many kinds of pipelines, Kanban is just one of them. It may be the main one they built it for (that's fine) but it's not the only useful pipeline.
Yes, I agree with most of this. But not all at the moment:
This exaggeration doesn't help. In practice, the "same thing" wont work for everyone, people work in different ways, or there will be hard constraints (eg many repos). For ex, it would suck for you to be forced NOT to use Github Milestones because others cant use it. you should use the best tool available to you. The good news is there's not 1001 ways, in practice there will be maybe 2-3 practical, effective ways of implementing 1 model. For example, I think we've identified the four most reasonable ways of handling "Roadmap Milestones" (or "Groups of Tasks") with Github features:
This is why i keep reminding to distinguish "Model" (the protocol, the abstractions) from "Mechanics" (how they're implemented). The important part is to agree on the model everywhere, and to make it clear how the model is implemented in each project. In practice the implementations will be almost the same everywhere but differ slightly.
Nobody is suggesting this. Everyone wants to find agreement on the model.
No, the "other pipeline" in question is just a different model being implemented for a different purpose. The blog writing pipeline has a different model that is orthogonal (they serve different purposes, they are not in conflict, and can therefore co-exist with) the kanban pipeline. Both models here are useful. And the blog should conform to "what we do everywhere" (i.e. use a kanban pipeline). But that doesn't mean people working on the blog cannot use waffle to serve another purpose too, with another board. (waffle != kanban. waffle is a tool to implement any pipelines). |
|
Thanks for the comments @jbenet, and especially defining the terms "model" and "mechanics". Surely, we've been using different definitions and I think to move the discussion to the right direction, let's focus on defining the terminology and "the model".
It finally makes total sense :) I'll be working on a document today to write this up, will refer the PR once there's something to show. As for the terminology: The complete description of what the steps and entities are, how they interact and how it's implemented is called the process. This should be collected, fully, in one document that works as our "source of truth" and as a manual for anyone wishing to opt-in to the "IPFS Project Management Process". Can we agree on that?
Fully agreed that the model should be described without a reference to a specific product or product features.
I think this is where the confusion, and difference in our terminology, comes from. I see the model as the description of what it is as in the entities and abstract concepts. How they interact I see as part of the mechanics, but for the sake of being on the same level, I'll use your definition in future communication, ie. include the interactions as part of the model.
As mentioned above, I see the mechanics as the way the pieces interact, ie. how ideas travel through the process into a release, the tools for each step in a pipeline (tools as in "a signal board" or "Backlog", not as in "Github"). For the sake of clarity, can we refer to this part as the implementation? The document I plan to write would include, then, the following: IPFS Project Management Process
Would that capture everything we have discussed so far, and result in what we're trying to achieve here?
I might back out from my current position on this as we move further, but my experience and instinct says this is the source of a lot of the information duplication we do. What I see as the main challenge to pull PM together as of today is that we have the (same) information in: documents in repos (files), PRs, issues, another document somewhere else than GH, the same issue/topic in multiple repos, etc. This leads to a lot of confusion as to what is the "source of truth" and where I can find it. We end up writing the information across repos (think our current Sprint process, or the quarterly planning process) and it takes a lot of overhead, when the time for planning comes, to gather all that information, present it and then go back and break it back to the format that suits a particular project or repository. An ideal way is to have that information to live in one place, and in a common format, from where it can be can be used and processed in multiple ways. As such my ultimate goal here is to make sure we produce that information only once and paying attention to how (format) and where (location) that information is collected is crucial. That is, to make sure the different users (views) can easily grab the information, regardless of the tool. There's one tool that we can't and won't get rid off, and that is Github. So whatever the implementation details are, we need to look at it through that lens. This might not add value to the conversation as of now and leans on the side of mechanics, but I'll try to clarify on this particular topic. Why I'm strongly proposing the use of Milestones vs. Label is not because "that's how I use it" but because I've been playing with the challenge of our information management the past weeks, trying to see how different pieces can be tied together in a coherent format that address the numerous constraints we have around the project: the multitude of projects and repositories we have, the open sourceness of the project, community participation, level of access, the fact that Github will be the tooling and information platform, the "director level" requirements as to what information needs to be displayed and to who, and flexibility for individual project leads. I understand the need to decouple the model from the implementation, but at the same time one can't design it all in isolation due to the already known constraints. Again, agreed that we should write the model down as an abstract-level description without assumptions of tooling. So, perhaps, it might feel like I'm jumping the gun with the implementation details without having the model defined and as such feel like it's an unfoundedly strong or subjective opinion, but I hope you keep in mind there has been work and thought behind it: I've read through the current documents, listened to the requirements and constraints from all sides, tested various combinations of structuring the information as well as evaluated numerous tools and features. I don't voice my opinion strongly unless I have something more than a gut feeling to back it up. Project management, and processes in general, are one of those funny things that feel counter-intuitive to an engineer as we're used to having a spec and then implement it. But at least to my experience, it usually starts from writing down what you do now to describe it and then iterate, ie. implement it as you design. It's a lot easier to tie the abstract concepts and words like "Backlog" or "Pipeline" together when you have something tangible, something "humane". The other way around it's... not as easy :) That all being said, this has been and still is a hugely valuable discussion and I think we'll converge soon. Whatever comes out of it will be better than what we have now :) <3 PS. should we change the issue title to "Project Management Process discussion" to reflect the discussion better? |
👍 ! Definitely a more accurate description. Lots of great conversation here, and I agree, a very, very valuable discussion. From the above conversation I believe a few things can be established:
Because we are till-death-do-we-part tied with Github, I recommend that the first place to start would be creating a labeling system that is consistent across the company. I want to air on the side of small implementations for process improvements with PM, which allows us to ditching things if they don’t work. I think that starting with something “small” like standardizing a methodology around how we use labels will be infinity useful, regardless of what higher mechanics we deploy (Waffle, etc.) I don’t think that we are going to be able to hammer out the entirety of the model/mechanics and be able to implement it fully in one fell swoop. I want to advocate on behalf of starting with something small, and then growing to adapt based on the needs/results of smaller implementations. What are the biggest concerns with this process or approach? |
|
@jbenet @em-ly @diasdavid please see #131 for a working document to define the project management process. All input is welcome! |
haadcode
changed the title
|
@flyingzumwalt I think you would have some valuable comments to this discussion and feedback on the document? |
|
This discussion is great and I like the document @haadcode is putting together. I've added a few comments there. My priorities:
I think you're on the right path for covering all of those things. |
|
When it comes to implementing this process, I'm curious to compare & contrast a kanban waffle board that uses Github milestones against @diasdavid's waffle board. I'm having trouble imagining the strengths & weaknesses of each. @haadcode would you be willing to set up a kanban-style waffle board for orbit so we can compare the two options? |
|
@flyingzumwalt it is really important to take into account that js-ipfs and libp2p are dozens of modules, while orbit is a repo. @jbenet documented really well how we should think about project management (define common language) and them map it to the tools we have available to manage each project - #125 (comment) |
|
@diasdavid I agree. My personal preference is to allow as much flexibility as possible when it comes to the implementation (aka. mechanics, etc.), providing only guidelines and examples of suggested approaches. As long as the community can understand your roadmap and you're able to fit the overall PM process, I think it's healthy to adapt and experiment on the mechanics. Why I asked @haadcode to make that waffleboard: Your waffle board provides a great example of what it looks like to take the labels-as-milestones approach, which does seem to fit with your constraints. I'm curious to see alternative approaches because it
|
|
@diasdavid would be awful for you to have both:
i made the default one be the kanban one as it's the one likely to be required by everything else. i could imagine similarly:
Thoughts for making it possible to have the same kanban board expectations across the projects, while also giving projects the flexibility of using their own ad-hoc pipelines where relevant. |
|
I wouldn't call it awful, but I don't imagine anyone opening two waffles to understand what issues from the current milestone (info available in waffle 2) are not in progress in the (info available waffle 1). |
|
Loads to reply to here, great comments and discussion everyone! I'll get back to this in more detail tomorrow. Had a quick chat face-to-face about this with @whyrusleeping the other day and one key thing we both mentioned and I think should be mentioned here: ideally, all information and as many parts of this process should be automated. Don't know yet how exactly, but ideally we do as little manual work as needed. Something for everyone to keep in mind as we go further with the tooling. Where the labels-as-milestones approach breaks, if used with Waffle, is when you move an issue down the pipeline, "milestone 1" becomes "in progress" and it loses its milestone information.
"Orbit" is a bunch of repos, not one. As such, the same constraints apply as for js-ipfs: no issues, labels or milestones across repos. However, I found a tool to do this and the workflow is quite nice being integrated with CI, see here: https://github.com/haadcode/labels-and-milestones. What it allows you to do is to have one definition file and sync all labels and milestones across multiple repos. |
|
@haadcode @em-ly @jbenet @flyingzumwalt I want to point you to this comment by @philschatz about another Open Source Kanban board that interfaces fluently with GitHub. @philschatz has been amazing to work with with Octokat.js, and he made a really nice post here about how to use his board, so I think it's worth bringing up. I'm not sure of the current status of our PM discussion, but this might be well worth putting our time into. Definitely worth at least checking out, as I'm not convinced waffle is the best. Please open elsewhere if we feel that would be best. |
|
@RichardLitt fully agreed, gh-board looks pretty cool, there was already some discussion around it on the PR for the PM document (which I can't find right now) |
flyingzumwalt
modified the milestones:
PM Process Implemented Across all IPFS Projects,
Ratify New Project Management Model
|
FYI: I'm going to close this issue when we've finished Milestone 4 https://github.com/ipfs/pm/milestone/4 |
ghost
removed
the
@diasdavid started the process on js-ipfs with https://waffle.io/ipfs/js-ipfs.
Let's take a step back here and go through the steps:
Re. the board, and from what I understand, you created the Milestone specific "Backlogs" to make sure you have an easy-to-grasp view to the different milestones and tasks in them. This, however, goes against the concept of the board being a pipeline and as such, it'll be better if all items were in one Backlog. You can filter the board view based on milestones (top right corner) to get the view you wish. You can then sort them in the Backlog column (top-to-bottom order) as per the milestone and priority of each goal/issue.
If needed, you can break down the Backlog into "Icebox" (things that are not going into development any time soon, other/misc issues) and "Backlog" to contain eg. the goals for Q3.
I would also add "Blocked" step to the pipeline to make sure we can surface blocked goals as easily as possible.
@diasdavid does all that make sense?
The text was updated successfully, but these errors were encountered: