2023.12.13

[seanmcilroy29]

---

2023.12.13 Agenda/Minutes

---

Time: Bi-weekly @ 1600 (GMT) - See the time in your timezone

• Co-Chair - Marco Valtas (Thoughtsworks)
• Co-Chair - Chris Xie- (Futurewei)
• Convener – Sean Mcilroy (Linux Foundation)

Antitrust Policy

Joint Development Foundation meetings may involve participation by industry competitors, and the Joint Development Foundation intends to conduct all of its activities in accordance with applicable antitrust and competition laws. It is, therefore, extremely important that attendees adhere to meeting agendas and be aware of and not participate in any activities that are prohibited under applicable US state, federal or foreign antitrust and competition laws.

If you have questions about these matters, please contact your company counsel or counsel to the Joint Development Foundation, DLA Piper.

Recordings

WG agreed to record all Meetings. This meeting recording will be available until the next scheduled meeting.
Meeting recording link

Roll Call

Please add 'Attended' to this issue during the meeting to denote attendance.

Any untracked attendees will be added by the GSF team below:

• Full Name, Affiliation, (optional) GitHub username

Agenda

• Approve agenda
• Approve previous Meeting Minutes

Introductions

• Any new members?

PR submissions

• Review - Dev: Fixed hyperlinks #15
  • Software_Carbon_Efficiency_Rating/Software_Carbon_Efficiency_Rating_Specification

Group agreed to merge PR

Submitted Issues

• The scope of SCER specification: would SCER address the specifications of data collection, and the resulted rating publication? #14
• Should SaaS be part of the software categorization?  #10
• Should these aspects of categorizing software be OR or AND relationship?  #11
• Benchmarking #12
• Examples for SCER Reference Applications and Technical Integration & Visualisation #1
• Review Spec

Summary of Spec Review

Software development and merging changes.

• Tammy and Sean discuss changes to the spec review, with Sean approving the final initial draft and merging a feature branch into the main branch.
  AI: Sean to resolve PR conflicts

Software energy consumption ratings.

• Tammy expresses concern about software ratings being affected by hardware variations.
• Tammy and Chris discuss the inconsistency in energy efficiency across different hardware manufacturers, with Chris suggesting that the issue is addressed in the spec but needs to be refreshed.
• Tammy and Chris discuss the importance of standardising test environments for software ratings, with Chris suggesting a baseline hardware setup.

Standardizing software testing environments.

• Chris suggests standardizing the test environment to remove variability in energy consumption and carbon emissions.
• Chris discusses the feasibility of the Green Zone Foundation's standardization efforts and the potential for other organizations to duplicate their testing environment.
• Oracle and Microsoft adopt similar testing standards for machine learning performance despite wide variations in software capabilities.

Software categorization and commonality.

• Chris proposes standardizing on standard components of education to create domain-specific software specifications.
• Tammy and Chris discuss standard software categorization and its relationship to performance.
• Chris explains their process for defining commonalities in software categorization, using examples from different industries.

Benchmark certification for software.

• Chris defines benchmark certification as a standardized workload or test on software, with consistent test environment specifications for comparison.
• Chirs explains standardized certification process for music streaming services.
• Chris and Tammy discuss potential concerns with the rating system, with Chris suggesting a standard platform for measuring hardware performance.

Standardizing GPU performance measurement.

• Tammy and Sean discuss the proposed baseline for measuring GPU performance, with Tammy expressing optimism and Sean suggesting a 15-minute presentation at the next standards meeting.
• Chris proposes creating an open-source product for crowd-sourced testing and rating of software, potentially leading to a governing body for GSF.
• The idea is to provide an intuitive, non-numerical way for people to evaluate software based on their own experiences and opinions.

Standardizing and certifying software performance metrics.

• Tammy and Chris discuss potential approaches for rating and approving energy-efficient products, including a community-based approach and a more formalized body to govern ratings.
• Chris suggests identifying common components in upcoming events to improve efficiency, while Tammy compares the situation to the SCI specification for measurement, noting that it's complex and requires expertise.
• The group discusses the need for a conformance program to validate scores in the context of a hypothetical "go cup" for computing community.
• Chris explains two approaches to getting a trustworthy number for a product's performance: one through official testing by a highway authority, and the other through customer-generated data using a benchmarking tool.

Improving a technical document with community input.

• Sean proposes bringing the baseline to the standards working group for review and feedback.
• Chris suggests collecting open questions and addressing them as a community effort.

AOB

• Member submissions

Future meeting Agenda submissions

• If you have any Agenda submissions, please let Sean know via email [email protected] or open an Issue

Next Meeting

• 10th Jan @ 9 am P/T
• Proposal to cancel 27th Dec meeting

Adjourn

• Motion to adjourn

---

Meeting Action Items / Standing Agenda / Future Agenda submissions

• Add here

[tmcclell]

Attended

[chrisxie-fw]

attended.