Piraeus Annual Review Evaluation and next steps

[TheFoxAtWork]

Related: https://github.com/cncf/toc/pull/1119/files. & #147

I am the TOC member assigned to conduct the Evaluation of Piraeus' Annual Review. At this time the TOC is requesting a roadmap that details the dates and actions for the project to improve its overall health and continued progress towards incubation. This roadmap should address the findings below with anticipated dates of resolution. The Roadmap is due to the TOC by January 31st 2024 and may be provided as a comment on this issue within the TOC repo and tagging @TheFoxAtWork.

Please note the findings listed below are not all encompassing and are intended to focus specifically on bringing the project into alignment with expectations of a Sandbox project. For more information, please refer to the TOC Archival process.

The following were findings from the evaluation of the Annual Review:

• Project needs to adopt the CNCF code of conduct. The governance file states it is aligned with the CNCF code of conduct however the CoC in the repository is the contributor covenant. Currently, TAG Contributor Strategy recommends projects link directly to the CNCF CoC within their own CoC file so they inherit any changes made in the future.
• While there are new contributors to the project, it does not appear to be achieving the anticipated velocity to make substantial progress towards the stated goals and towards incubation.
• While the project's README indicated active discussions take place in Slack, a quick review of Piraeus slack shows very little discussion on going beyond periodic requests for assistance in the default channel (about once a week). This is not indicative of a growing community.
• The project does not appear to hold meetings to engage in synchronous discussions with contributors, adopters, community members, perform project planning, or any other content. If meetings or discussions are happening in other channels, or platforms, the project must update its documentation to reflect these discoverable and public communication mediums.
• The project does not appear to engage in any long term planning.
• While development is ongoing, there does not appear to be additional planning towards future releases, no open issues tracking progress or discussions on outstanding areas of improvement, etc. It does not appear tags or milestones are used.
• issues with the project have been open for an extensive period of time with discussions, if they occurred, remaining unresolved.

Additional recommendations for the project:

• review the definition of adopters in the TOC repo and update the adopters listing for this project to ensure those types of adopters are properly captured. A few of the adopters listed do not appear to meet this definition.
• Engage with TAG Contributor Strategy to explore opportunities to increase the pool of contributors, identify potential emerging maintainers, improve retention and increase activity of existing contributors
• Engage with TAG Storage to determine any additional insights and recommendations the TAG may have in bringing closure to its open goals and work areas.

[TheFoxAtWork]

@WanzenBug @alexzhc @JoelColledge @rck checking in here on the status of this.

[Philipp-Reisner]

• Project needs to adopt the CNCF code of conduct. The governance file states it is aligned with the CNCF code of conduct however the CoC in the repository is the contributor covenant. Currently, TAG Contributor Strategy recommends projects link directly to the CNCF CoC within their own CoC file so they inherit any changes made in the future.

Addressed with 99a926d

• While there are new contributors to the project, it does not appear to be achieving the anticipated velocity to make substantial progress towards the stated goals and towards incubation.

Thank you for pointing this out. Our expectation with making this a CNCF (sandbox) project is to increase the velocity.

• While the project's README indicated active discussions take place in Slack, a quick review of Piraeus slack shows very little discussion on going beyond periodic requests for assistance in the default channel (about once a week). This is not indicative of a growing community.

Thank you for pointing this out. That is a consequence of the README being outdated. Addressed that with d77d8cf

• The project does not appear to hold meetings to engage in synchronous discussions with contributors, adopters, community members, perform project planning, or any other content. If meetings or discussions are happening in other channels, or platforms, the project must update its documentation to reflect these discoverable and public communication mediums.

Piraeus Datastore is just a glue component between LINSTOR and Kubernetes. People interested in Piraeus Datastore are also interested in LINSTOR and DRBD. LINBIT organizes about 4 community meetings every Year.
In these meetings we regularly have updates about the Pireaus Operator and a synchronous live discussion in a Zoom meeting that is also live streamed via YouTube.

Addressed with d77d8cf

• The project does not appear to engage in any long term planning.

Piraeus Datastore’s long-term plan is to stay current with Kubernetes, LINSTOR, and DRBD developments. I think that is obvious for people looking at it from a technical perspective.

• While development is ongoing, there does not appear to be additional planning towards future releases, no open issues tracking progress or discussions on outstanding areas of improvement, etc. It does not appear tags or milestones are used.

Frequently, Piraeus Datastore’s releases are triggered by LINSTOR releases. I think that is obvious for people looking at it from a technical perspective.

• issues with the project have been open for an extensive period of time with discussions, if they occurred, remaining unresolved.

We like to keep issues open until the underlying issues are resolved or the original author marks the issue as resolved. Oftentimes, this means the issue stays open after initial activity: we try to resolve all reported issues, but we sometimes have a hard time pinpointing the reported issues. In that case, we are dependent on the creator to provide enough information.

Unlike many Kubernetes-related projects, we do not deploy a bot that automatically closes issues after a set period of inactivity. If that is desired, we can set it up.

PS: We love creating great software, but we need help with foundation bureaucracy.

[TheFoxAtWork]

@Philipp-Reisner Thank you for taking the time to respond and thank you for quickly addressing those recommendations.

Our expectation with making this a CNCF (sandbox) project is to increase the velocity.

This is a reasonable, Sandbox projects are expected to experiment, innovate, and mature towards incubation. However projects are expected to take steps towards incubation, build a sustainable community of contributors and maintainers, and continue to mature the technology alongside their development practices. For some projects, that velocity may be slower due to dependencies or alignment with other projects - this is fine. We want to ensure that our sandbox projects are set up for success and heading in the right direction, thus our periodic check-ins (such as what spawned this Issue).

Piraeus Datastore’s long-term plan is to stay current with Kubernetes, LINSTOR, and DRBD developments. I think that is obvious for people looking at it from a technical perspective.

I genuinely believe every project in the CNCF is intending to stay current with developments in the projects they are dependent upon or interoperate with. The TOC leverages long-term planning as an indicator of evolving project maturity.

In some cases, potential contributors look at project planning to understand if a particular feature or proposal is already planned to be worked, underway, or delayed until a future release. Visibility and transparency into where the project is headed and the open work before them is a good mechanism to attract potential contributors and adopters.

Frequently, Piraeus Datastore’s releases are triggered by LINSTOR releases. I think that is obvious for people looking at it from a technical perspective.

I would recommend adding additional documentation describing the release process for the project so newly come contributors or interested community members can learn about the process and support the maintainers in conducting releases for the project. Kubernetes SIG-Release has a lot of good information for the K8s project does this, it is a heavy for a project like Piraeus however there may be good information for the project to use.

We like to keep issues open until the underlying issues are resolved or the original author marks the issue as resolved. Oftentimes, this means the issue stays open after initial activity: we try to resolve all reported issues, but we sometimes have a hard time pinpointing the reported issues. In that case, we are dependent on the creator to provide enough information.

Makes complete sense, does Piraeus plan to work on all open issues or are some slated for work after certain LINSTOR releases make a feature set or capability available? Its these kinds of planning and engagement activities we look for as indicators of maturity towards incubation. I would recommend adding some clarity to the contributor guide around this to set expectations with individuals submitting issues and PRs (there is a lot of good content in the contributing guide but additional clarity in how the project manages issues and PRs for planning releases is always beneficial).

Unlike many Kubernetes-related projects, we do not deploy a bot that automatically closes issues after a set period of inactivity. If that is desired, we can set it up.

That is up to the project to determine. If you are facing significant issue sprawl and there are known issues that aren't going to be worked, an inactivity bot can take some of the burden off maintainers in closing them, particularly if the submitter is no longer responsive. It just depends on how the project chooses to perform and subsequently document their issue and PR management activities.

PS: We love creating great software, but we need help with foundation bureaucracy.

Fantastic Piraeus love making great software! Amazing cloud native projects have wonderful documentation to go with their great software, both for their current and potential contributors looking to get more involved in the project, as well as for adopters and their organizations looking to understand more about the project, its processes, how it functions, etc. This can support adopters in making informed decisions about the project before committing to a cycle of development, integration and testing, and finally production deployment.

Given your request for assistance in understanding the criteria and expectations for projects in the CNCF, I would recommend reaching out to TAG Storage and participating in one of their meetings as well as reaching out to TAG Contributor Strategy to talk through some of potential discoverability and visibility recommendation which remove ambiguity from the project documentation and processes. CC TAG Storage Chairs: @chira001 , @raffaelespazzoli , @xing-yang & TAG CS Chairs: @jberkus , @CathPag , @geekygirldawn

Regarding the obvious nature of discerning project activity, direction, and planning:

Although these project areas may be obvious to maintainers, it may not be to individuals who are not involved in the day-to-day functions and operations of the project, individuals like users or other engineers looking to contribute - particularly those individuals interested in contributing to a project but are hampered from doing so by the lack of specificity or explicitness in their documentation on how to engage.

I didn't see you listed as a maintainer for the project @Philipp-Reisner, if this is a role you have with the project would you also ensure the MAINTAINERS.md file is up-to-date based on the project's governance?

[xing-yang]

Hi @Philipp-Reisner, let us know if you want to come to one of TAG Storage meetings and discuss about Piraeus.
cc @chira001 @raffaelespazzoli

[TheFoxAtWork]

@rck @wazenbug @JoelColledge @alexzhc
The TOC is requesting an update from the project on its current lack of activity and alignment with CNCF project expectations. Is this project still active? it appears only components supporting LINSTOR and DRBD are active.

Please provide an update no later than Friday December 6th to allow the TOC to make an informed decision regarding the project's health and potential for archival. CC @mrbobbytables

[WanzenBug]

Sorry for the late response.

The TOC is requesting an update from the project on its current lack of activity and alignment with CNCF project expectations. Is this project still active?

Yes it is still active. Just last week, we launched an update to our documentation page: https://piraeus.io/docs/
We've also had a presence at Open Source Summit Europe, including a talk on how we integrate Kustomize with our Piraeus Operator.

it appears only components supporting LINSTOR and DRBD are active.

Sorry to be snide here, but this is like saying there has been not much activity in the Rook project apart from components supporting Ceph and NFS. Now of course we are talking about completely different project sizes here.

LINSTOR and DRBD are the core of Piraeus Datastore. It's what makes it run. So of course the main activity will be focused on them.