Storage system query support

[shanedsnyder]

Based off of #265 (thanks @rhc54 )

Work in progress as we sort out PMIX_STORAGE_TYPE attribute.

[kathrynmohror]

Please use emoji reactions ON THIS COMMENT to indicate your position on this proposal.

• You do not need to vote on every proposal
• If you have no opinion, don't vote - that is also useful data
• If you've already commented on this issue, please still vote so
  we know your current thoughts
• Not all proposals solve exactly the same problem, so we may end
  up accepting proposals that appear to have some overlap
  This is not a binding majority-rule vote, but it will be a very
  significant input into the corresponding ASC decision.

Here are the meanings for the emojis:

• Hooray or Rocket: I support this so strongly that I
  want to be an advocate for it
• Heart: I think this is an ideal solution
• Thumbs up: I'd be happy with this solution
• Confused: I'd rather we not do this, but I can tolerate it
• Thumbs down: I'd be actively unhappy, and may even consider
  other technologies instead
  If you want to explain in more detail, feel free to add another
  comment, but please also vote on this comment.

[SteVwonder]

A few inline comments below. One general question is what error codes will be returned if the storage system doesn't know how to respond to a specific BW, Capacity, or IOPs query?

[shanedsnyder]

A few inline comments below. One general question is what error codes will be returned if the storage system doesn't know how to respond to a specific BW, Capacity, or IOPs query?

@SteVwonder We are just assuming that storage systems and/or the PMIx server will return PMIX_ERR_NOT_SUPPORTED for unsupported storage system attributes similar to how different PMIx implementations can return this for unsupported function calls.

[rhc54]

@shanedsnyder Please remember to add the "Signed-off-by: Name <email>" line to your commit(s) so they can be committed!

[jjhursey]

ASC Meeting Oct. 1, 2020

• Good feedback and discussion on this ticket. WG will take this back and update this PR.
• Decided to use the "Provisional Acceptance" timeline/process for this ticket. (See governance document for exact procedure)
  • This means that the authors may continue to work on this PR and re-present it in the next quarterly meeting.
  • Well before the next meeting post another straw poll and PDF once it is ready for review.
  • Chairs will re-add the "Eligible" label and add it to the agenda.
  • In the next meeting they may present another reading and, if successful, vote.
  • If all goes well, then this would be included in the next minor release marked as "provisional"

[jjhursey]

1Q 2021 ASC Meeting (Feb. 18, 2021)

• Revision Exception vote passed: 10 yes / 0 no / 2 abstain / 2 absent
• This PR enters a 2 week comment period. If no objection then it will be merged for the next minor release.

[jjhursey]

It looks like some of the commits were not signed off. You will need to interactive rebase and update the commit messages. That'll like rewrite history, but that's ok. Let me know if you need a hand with that.

Once that is done then we will need to add the provisional markers (pending PR #338), but the release managers for v4 can help with that.

[rhc54]

@jjhursey We can't accept this yet because there is no prototype implementation - we still have not seen the final PR for OpenPMIx. AFAIK, that is a requirement even for provisional acceptance, yes?

[jjhursey]

An implementation is not required (looking at 1.3.3 of the governance doc), but anyone can request one as a reason to hold the merge. I'm ok with holding merging this until we get one.

[rhc54]

Somehow, that just seems like an oversight - nobody should be able to put something in the Standard that has never been implemented, at least as a prototype. We have no way of knowing if the proposal is even feasible. Should definitely amend that section to correct this.

[jjhursey]

I'm open to it. Do you want to file an issue in the governance repo for discussion?

[shanedsnyder]

It looks like some of the commits were not signed off. You will need to interactive rebase and update the commit messages. That'll like rewrite history, but that's ok. Let me know if you need a hand with that.

Just fixed this.

[shanedsnyder]

FYI, what I have for initial implementation is here: openpmix/openpmix#2086

Do we need implementation for every single attribute? Some of these aren't going to be practical in the short-term... The bandwidth and IOPS attributes obviously are going to require storage system APIs to provide the info -- Lustre has said that's something they could provide in the future, but I don't know that anything is available now. Similarly, the attributes related to storage medium and storage accessibility are going to require system-specific knowledge that I'm not sure how we'd support in OpenPMIx.

I understand the desire to want things implemented before accepting into the standard just really wish this issue was raised sooner, as we've potentially wasted a lot of time now. If we need to focus on things that can be implemented today, I suspect we're going to be really limited in what we can provide.

[rhc54]

I don't think you have to implement support for everything. I just want to see that the concept has been vetted. What you posted should suffice for that purpose, and I appreciate it.

[jjhursey]

I think there is a balance we need to strike with implementation requirements. Thanks Ralph for opening https://github.com/pmix/pmix-standard/issues/341 for discussion. We have a monthly call this week I'll add it to the agenda for discussion.

[rhc54]

@pmix/asc-chairs @pmix/asc-secretaries I believe this is ready to be committed, having met the review/vote as well as the implementation requirements I had requested.

Thanks @shanedsnyder for all the hard work!