2024.01.10

[seanmcilroy29]

---

2024.01.10 Agenda/Minutes

---

Time: Bi-weekly @ 1600 (GMT) - See the time in your timezone

• Co-Chair - Marco Valtas (Thoughtsworks)
• Co-Chair - Chris Xie- (Futurewei)
• Convener – Sean Mcilroy (Linux Foundation)

Antitrust Policy

Joint Development Foundation meetings may involve participation by industry competitors, and the Joint Development Foundation intends to conduct all of its activities in accordance with applicable antitrust and competition laws. It is, therefore, extremely important that attendees adhere to meeting agendas and be aware of and not participate in any activities that are prohibited under applicable US state, federal or foreign antitrust and competition laws.

If you have questions about these matters, please contact your company counsel or counsel to the Joint Development Foundation, DLA Piper.

Recordings

WG agreed to record all Meetings. This meeting recording will be available until the next scheduled meeting.
Meeting recording link

Roll Call

Please add 'Attended' to this issue during the meeting to denote attendance.

Any untracked attendees will be added by the GSF team below:

• Full Name, Affiliation, (optional) GitHub username

Agenda

• Approve agenda
• Approve previous Meeting Minutes

Introductions

• Any new members?

Submitted Issues

• The scope of SCER specification: would SCER address the specifications of data collection, and the resulted rating publication? #14
• Should SaaS be part of the software categorization?  #10
• Should these aspects of categorizing software be OR or AND relationship?  #11
• Benchmarking #12
• Examples for SCER Reference Applications and Technical Integration & Visualisation #1

Summary

Spec Development Proposal and focus areas

Software energy consumption ratings.

• Concern about software ratings being affected by hardware variations.
• Inconsistency in energy efficiency across different hardware manufacturers should be addressed in the spec.
• Importance of standardising test environments for software ratings, including a baseline hardware setup.

Standardizing software testing environments.

• Standardizing the test environment to remove variability in energy consumption and carbon emissions.
• Feasibility of the GSF standardisation efforts and the potential for other organisations to duplicate their testing environment.
• Oracle and Microsoft adopt similar testing standards for machine learning performance despite wide variations in software capabilities.

Software categorization and commonality.

• Standardizing standard components of education to create domain-specific software specifications.
• Standard software categorization and its relationship to performance.
• Process for defining commonalities in software categorization, using examples from different industries.

Benchmark certification for software.

• Define benchmark certification as a standardized workload or test on software, with consistent test environment specifications for comparison.
• Standardized certification process for music streaming services.
• Potential concerns with the rating system, with a standard platform for measuring hardware performance.

Standardizing GPU performance measurement.

• Proposed baseline for measuring GPU performance
• Creating an open-source product for crowd-sourced testing and rating of software, potentially leading to a governing body for GSF. The idea is to provide an intuitive, non-numerical way for people to evaluate software based on their experiences and opinions.

Standardizing and certifying software performance metrics.

• Potential approaches for rating and approving energy-efficient products, including a community-based approach and a more formalised body to govern ratings.
• Identifying standard components in upcoming events to improve efficiency while comparing the situation to the SCI specification for measurement, noting that it's complex and requires expertise.
• The need for a conformance program to validate scores in the context of a hypothetical "go cup" for the computing community.
• Two approaches to getting a trustworthy number for a product's performance: official testing by a highway authority and customer-generated data using a benchmarking tool.

Spec Review

• Proposal to discuss the Spec updates at the next WG meeting?

AOB

• Member submissions

Future meeting Agenda submissions

• If you have any Agenda submissions, please let Sean know via email [email protected] or open an Issue

Next Meeting

• 24th Jan @ 9 am P/T

Adjourn

• Motion to adjourn

---

Meeting Action Items / Standing Agenda / Future Agenda submissions

• Add here

[seanmcilroy29]

Approved