-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[RFC] New Labels for OpenSearch Core Repo #10566
Comments
Thanks for writing this up, @macohen! A couple of thoughts.... 1/ For "No less than once per two weeks, untriaged issues should be reviewed" I think the cadence needs to be once a week, not twice a week? That's what we've been doing for the last two years. The reason we don't want to wait two weeks is that, even through we ask folks to report potential security issues AWS/Amazon Security via our vulnerability reporting page or directly via email to [email protected], we still have had multiple instances of reports getting posted to github. We really don't want to wait 2 weeks with an issue out there without someone taking action. 2/ Many teams (shout out Security!) have moved to a public triaging mechanism. What's the plan to move Core to a more open triaging model? (@krisfreedain poke) 3/ You say "This RFC will also become a markdown file in the OpenSearch repo and we will add a link to the issue templates in the repo so issue creators can reference this process to help it go smoother." Can we also add something to .github that describes all the project labels, as well as something that describes the repo specific labels? That way you can just "include" the project label file for the broader labels. |
100%
right now, we have part of core in this model. The Search Relevance Backlog & Triage is public. We review this board which has issues labeled "Search" https://github.com/orgs/opensearch-project/projects/45. Turns out to be anywhere from 0-10 issues per week. Core overall has many more right now. In the last public meeting, we realized that we talk more about search/core/search pipelines issues than actual relevance topics. I was thinking about broadening the scope of that meeting to talk core search overall, but would like feedback. There are many more untriaged issues in core than we can handle in one meeting, which is why splitting these labels was a good idea (this is really an open version of an idea/document @anasalkouz wrote up that we discussed with @yigithub (i get this login now after looking at it a bunch - funny), @kkhatua and others). We first discussed this, then put out the labels, and then want the feedback so we can refine. Chipping away at what's there in a public meeting would be good, but we also want to get the community involved in big picture discussions in those meetings. Happy to discuss this in public next Wednesday at 9AM PT: https://www.meetup.com/opensearch/events/295393206/.
Yes! Sounds great. |
Yes! thanks @CEHENKLE & @macohen - so far we have public meetings for 'Security' (shout-out to @scrawfor99, @peternied, & @davidlago for being the pioneers), 'Search Relevance' from @macohen & team, 'Dashboards' from @joshuarrrr & team, as well as 'ml-commons' from @ylwu-amzn & team. We'll continue to chip away at each of the dev teams and make each of them public - sooner rather than later. |
Thanks @macohen. In order to truly move to a component model and reduce the high load on core maintainers/core oncalls to triage issues, can we get issue originators to attach a Component/Area label when the issue is opened? Someone opening an issue has a decent idea on the area the issue is being opened for. For the issues where ownership is unclear, originators can leave it 'untriaged'. For the issues that start with an area/component, the first component owner can move the issue downstream to another component if the issue is found to belong to another component after initial analysis. This is a better model to scale our core operations. |
The main usage of the untriage label here is to mark everything as "new" so we don't miss potential security issues. The rest of it could be handled other ways. I think we could do two things to simplify the process. |
Great issue @macohen, I'll have to join one of your session to see what I can learn from from how you get things done. A couple of thoughts on acknowledgement vs accepting an issue. From the security triage we've got a multiphase process that works well for us, issues are always labeled untriaged automatically on creation/reopen. Issues are marked 'triaged' when they are deemed actionable by the team only during the triage meeting. We have a section on labeling that calls out this lifecycle https://github.com/opensearch-project/security/blob/main/TRIAGING.md#what-are-the-issue-labels-associated-with-triaging |
HI @macohen, this seems like a great change! As Peter mentioned, Security has a document we follow each time which can be helpful for keeping a consistent process. It also helps keep a consistent order of the meeting and avoid things running over. For the labels you proposed, I think the majority of them seem good. I was wondering at what scale these meetings were expected to take place. I am not sure how easy it will be for people who are not subject matter experts to identify the difference between an issue which should be assigned 'Area:Relevance' vs. 'Area:Capabilities' for example. I know that there are some ares where an expert can clarify this distinction, but I wanted to check whether you felt this would be better served with broader categories or think it would become too confusing? Either way is fine, I just wanted to raise this point. Overall, this is a really great idea. |
@scrawfor99 that's why we created the RFC! we need to balance the need to create a meeting for every label with focus and concentration. The search relevance meeting today is typically 10-15 minutes of triage and then deep dive into other topics with time for anyone to raise things they want to discuss. Expanding that meeting to all of core is way too much to cover anything but triage, IMO. Thanks for sharing the process doc @peternied! |
@Pallavi-AWS AFAIK sadly adding label requires commit rights on the repository (at least for now), so external contributors (as originators) won't be able to add those |
@macohen, thanks for the quick reply. Again, I think this is great. I guess it depends on who will be assigning the labels as well as how it will be used. I think the broader categories are quite good as-is but I am hesitant to add too many new labels because there are already so many. Without removing some existing ones I would probably vote for using the broader labels. Will we be able to remove the old labels we no longer use? I see for example we have Search as a label but maybe we can remove that one when we add the new etc.? If we can remove a lot of the old ones then I think that the new labels look good. |
@scrawfor99 I think maintenance of those labels belong to the repo maintainers themselves. Assigning labels could be done by anyone. Should making sure the issues/labels are assigned correctly belong to maintainers or others who can update labels? (@opensearch-project/triage members)? I think so. What do you think? BTW, I also think there should be an ability to change labels. We could generalize them or make them more specific as needed. Looking at this as a starting point. Also, I'll include a reference to the security triage process as an example of how to do it. I think repo/label owners should be able to find their own path, but I like using the first triage process as a good example. |
Oh. Also, to answer @CEHENKLE's question about moving core to a more public process, we do triage core issues with certain labels in the Search Relevance meeting because they automatically appear on the board we review: https://github.com/orgs/opensearch-project/projects/45 Because Core is so large, I think there's use in having a few of these meetings. cc: @getsaurabh02 , @anasalkouz , @yigithub , @kkhatua , @Pallavi-AWS |
Hi @macohen, thanks again for keeping this moving forward. I don't think GitHub will let people other than maintainers modify labels or assign/unassign them. As long as we have someway of keeping the number of labels used and the number of labels that exist in the project as a whole (the maintainers can do this like you said), I think we will be in good shape. I also would be incredibly in favor of more parts of the project adopting the Security repo's triaging model (or the similar version that you and Search do). |
@aalkouz, @krisfreedain, and I had a discussion I'm documenting here
Either way, we also will look for a maintainer per component/label. If there are any maintainer volunteers to take care of a specific label, please raise your hand here! Any thoughts, questions on these ideas? Volunteers? |
@mch2 will you be able to take care of the changes to our issue template to assign component labels at the time of opening an issue? We want to avoid the pileup of issues on core. Thanks. |
@macohen I'm a maintainer, but I don't If we had a simple majority of maintainers vote |
let's vote then, but also clarify what the vote is about. There also may be 1) a different way to slice the components up, 2) a gap in maintainers who do own those areas - I'd ask maintainers to review this based on merit - the usual way and see if any new folks qualify. My concern is that if everyone owns it then nobody does which seems to be the state we're in now. But, that means we need to adjust something. I'd like to simplify this down to a few options for a vote and get more maintainers to chime in on these. 1) do these component labels make sense? 2) do we have at least one maintainer who can be responsible for triaging and clarifying label assignments right now? 3) do we agree on the goal? - no issue has an untriaged label for more than one week and issues can be assigned a label to further categorize issues so action can be taken. cc maintainers: @abbashus @adnapibar @anasalkouz @andrross @Bukhtawar @CEHENKLE @dblock @dbwiddis @dreamer-89 @gbbafna @kartg @kotwanikunal @mch2 @msfroh @nknize @owaiskazi19 @peternied @reta @Rishikesh1159 @ryanbogan @sachinpkale @saratvemulapalli @setiah @shwetathareja @sohami @tlfeng @VachaShah |
I am not a maintainer but I have a mixed opinion regarding this issue. @macohen I agree with all of your points which can potentially lead us to the goal of |
@vikasvb90 I agree with you on diffusion of responsibilities. thanks for that. My intent is not to say that one and only one maintainer is responsible for each component, but I definitely see why it comes across that way. I wonder if a process where there are at least two maintainers who are the go-to for each label, but of course, any maintainer can assign labels as they judge. |
Having component labels just gives a good starting point for issue triaging as we scale. All maintainers will have access to all issues, it is not exclusive to maintainers from that component only. |
…labels in core for easier understanding of what's there Signed-off-by: Mark Cohen <[email protected]>
Closing as I believe we have created component labels. If there are any outstanding issues here please comment and/or reopen. Thanks! |
OpenSearch Core (https://github.com/opensearch-project/OpenSearch) is a large repository with many components. We want to make it more efficient and helpful to identify issues belonging to specific components, so we’re proposing some new labels to get more specific. This RFC is a place for people to comment on these labels. Are they effective? Do they mean something useful to the community? Are there missing lables? Too many labels?
What Do We Do With These Labels Anyway?
Across all repositories in opensearch-project, we have a process called “triage.” The goal of this process is to make sure repository owners and maintainers check new issues to understand them. First - we learn if the issue is security related (e.g. CVE), second - we understand if we have enough information to act on the issue, third - we re-label or respond on the issue. During triage we do not work on these issues, but instead just try to get the issue to a next step. Because the OpenSearch core repository (https://github.com/opensearch-project/OpenSearch) has so many different functions with different sets of contributors working on these functions, repo owners decided to treat these functions like separate repositories when triaging to get through the backlog faster and have more focus. There are a few maintainers that do triage in public (see https://meetup.com/OpenSearch to join) and there will be more coming.
Here is an example of the workflow for a new issue in the core repo.
TABLE 1
Questions for the Community
I plan on keeping this RFC open until 2023-11-11.
The text was updated successfully, but these errors were encountered: