Skip to content

SLIM Open Source Community Meeting

Jump to bottom
Kyongsik Yun edited this page Dec 12, 2024 · 88 revisions

December 12, 2024

Agenda

  • News & Activity

    • New Process Improvement: Continuous Delivery (#69)

      • GitHub link: [Continuous Delivery Issue #69]
      • Goals:
        • Iterate on feedback with the SLIM community
      • Next steps:
        • provide real world examples (slim-cli, slim-leaderboard -> pypi project)
        • provide Github Actions Workflow template

    • Open Source Web Presence: LLM-Based Content Generation

      • Context:

        • A significant number of science algorithm users may benefit more from this feature compared to project developers.
        • The LLM-based content generation feature of SLIM might add greater value than the best practices repository generation.
        • There is an evident demand among science team algorithm developers, especially more than CS-oriented developers.

      • Objectives:

        • Prototype SLIM CLI LLM Docusaurus generation process

      • Explore potential use cases:

  • Suggestions from the Community

    • Time zone for the meeting invite
    • Ready for external folks to join!
    • Jeff Leach: AWS event recap
      • NASA ongoing effort: Claude 3.5 for code generation for AWS Bedrock
      • More interested in open source code generation

  • Next steps:

    • provide documentation on downloading local models (big models like llama3.1 405b parameters)
    • short video for introduction, or mightydev presentation (install llama, slim-cli, etc., targeting local models) -> write a issue ticket
    • hard to run a model for scientists -> we should make it easier, more accessible.

parking lot

October 30, 2024

Agenda

  • News & Activity

    • SLIM Leaderboard overall scoring & nightly scanning repos.

    • HySDS website generated by docusaurus + genAI: https://github.com/yunks128/hysds.github.io

      • Originally SLIM CLI used to bootstrap new open source projects. e.g. starting with infusing tempalted best practices and guides.
      • Now prototpying usage of SLIM CLI and genAI to help generate content for open source websites and documentation.
      • Are there any other projects interested in joining SLIM approach of leverging genAI for content generation as well?
      • Generated quick start guide steps are misleading. Needs manual changes.

    • Will be presenting brief intro to SLIM to the ESA (European Space Agency) MAAP teams next week.

      • First exposure of SLIM to international community.

  • suggestions from community:

October 17, 2024

Agenda

  • News & Activity

    • Please note that the next meeting on October 31, 2024, will take place in room 168-532, instead of the usual 168-425, due to the Section 398 Halloween event.
    • results of ROSES F.7 OSTFL proposal awards
      • 14% success rate.
      • There was less funding to F.7 program areas and so they could not fund everything they had intended.
      • F-Prime got "selected". Was Rishi/SLIM part of this proposal?
      • HySDS got "selectable"
      • MMGIS got "selectable"
      • SDAP: not selected
      • AERIE: got "selectable"
    • ES2A RAPIDS: year 1 includes 0.15 FTE for SLIM involvement.

  • suggestions from community:

    • MAAP infusion efforts

      • Share new infusion and get suggestions from community
      • Infusion will be with new community of science users, and not just platform developers.
      • Many of the users are at UMD and international (e.g. ESA, CEOS)
      • We can clarify how SLIM is difference from TOPS.
        • How SLIM contributes to TOPS objectives.
        • e.g. How it contributes to permissive license?
      • As we engage with more community, would/should this SLIM community meeting be public? Would this slow down these bi-weekly meetings? It needs to scale out in a different way as we branch out.
        • Meeting notes and Slack (linked to AMMOS) are already public.
        • This will be more relevant as we engage with external-to-JPL projects.

    • NASA Software Slack org that is open

      • Would NASA AMMOS be willing to host this?
      • AMMOS Slack is currently free-tier (therefore limits to 10K messages rolling)
      • What about public github discussions that is open?
      • Slack is targeting a different velocity of discussion.

    • feedback on RAPIDS-related expectations.md and code of conduct, governance (work in progress) issue #173

      • How a project conducts open research?
      • existing code of conduct was mainly focusing on behavioral aspects.
      • How projects can communicate how researchers reuse code, attribution, and expectations of collaboration.
      • Cedric pitched this to Crawford. Should this be in a code of conduct, or expectations md?
      • Can we find other researchers who are conducting this with TOPS?
        • Should we post these questions on TOPS?

    • [New Process Improvement Need]: Continuous Delivery #69

      • potential next steps?
        • Finalize Repository and Naming Conventions
          • Several repository and naming conventions are outlined (e.g., PyPI, DockerHub, Maven).
          • Confirm and document the agreed-upon repository choices, especially for large datasets (Zenodo, Science Data Bank, etc.). Need to decide whether AWS CodeArtifact should be integrated or rejected (compared to PyPi?).
        • Automation for CI/CD Pipeline
          • The primary need is to automate the build and push processes to the chosen repositories. GitHub Actions/Workflows is the suggested tool.
          • Create and share reusable workflow templates for common repositories like PyPI, Maven, and DockerHub.
        • Handling Large Test Data
          • You may centralize smaller test datasets (<2GB) in cloud-based version control systems (VCS) like GitHub Releases, but larger datasets need a solution, possibly using S3 or DAAC for Earth data.
            • devs have tried splitting large files.
            • gitlfs backed by artifactory
            • Zenodo has repo limits of 50GB
            • TOPS has requirement for NASA to have a referencable software copy
          • Define a clear storage and retrieval strategy for both small and large test data, including cost considerations for S3.
          • Set up scripts or workflows to handle dataset uploading, retention, and versioning.
        • Guide Creation
          • A SLIM best practices guide is needed to compile all the aforementioned strategies, repository decisions, automation workflows, and versioning schemes.
          • Draft the continuous delivery guide, starting with the repository, automation, and versioning steps. This guide can be integrated into the repository's docs folder.
        • Continuous Testing Integration
          • Since the continuous delivery process also involves ensuring that packages are tested before release, integrating a continuous testing strategy will strengthen the CD process.
          • Build upon the recent Continuous Testing Guide ([New Best Practice Guide]: Continuous Testing Guide and Checklist #110) by linking the testing pipeline to the delivery pipeline, ensuring tests are automatically executed before artifacts are published.
      • Other projects have elements of this. Should we check with them? e.g. MIPL, NISAR, SWOT, OPERA, SMAP, etc.
      • This seems to be a large ticket. Should it be broken up into smaller items? different guides from CD, CI, CD
      • Prior approach is smaller concrete implementations and write guide for each.
      • How do we do full integration of the steps of CI, CD, CT, CD?
      • How to accommodate different approaches? Keep it at ghig level
      • Solicite inputs from Mighty Devs community?

October 3rd, 2024

Agenda

  • 5m News & Activity

    • Continuous testing plan best practice is infused into JPL's ROSA (Robot Operating System Agent)
      • Much thanks to Rob Royce
      • https://github.com/nasa-jpl/rosa/pull/24
      • how was PR initiated?
        • this infusion used the SLIM-CLI-generated best practice
        • CLI scans and generated testing plan, then submitted PR.
        • opportunity for CLI usage by external repo (outside of AMMOS) for best practices of using CLI.
          • action item: @KS - SLIM-CLI readme could include some of these examples.
          • @RobRoyce could also try the contribution back to SLIM.
          • this example highlights external community contribution.
    • @Sean usage of SLIM in PDS
      • image security also now being used in PDS
      • this has impacts of two-way contributions

  • 30m suggestions from community:

    • Guide on Code Security Scanning #148

      • https://github.com/NASA-AMMOS/slim/pull/148#issuecomment-2240048198
        • Jeff asked: "It would be helpful to get an example of how to use sonarCloud and sonarQube in the action to align with updated guidance from NASA MGSS. Some of this work my already have been performed by Elyssa but it would be good to have it documented here as well."
      • Public repos of MGSS accidentally had sonarcloud scanning enabled. e.g. FEI.
      • How generalized should the best practices guide be?
        • "how prescriptive should the recommendations be?"
        • sonarQube is a technology solution used at JPL. Not all repos may be using this.
        • The landscape of MGSS scanning guidance has shifted to more open source scanners.
        • Should the guide cite JPL-specific tools in use?
        • CMU has recommendations (report) on code scanning, with eye towards NASA AMMOS and with SLIM.
          • an CMU SEI person can contribute via SLIM to the code security scanning guide.
        • We can offer some options to people, and modify it for their own project-specific use.
          • Generic decision tree approach recommendation over technology selection.
        • We will need to keep these guides decoupled: JPL-internal projects (on JPL GHE) and public projects (on github)
          • JPL GHE has enabled github-actions so this helps to normalized the recommendations across internal and public repos.
        • Can we create PRs to help bootstrap projects?
          • common compliance
          • testing frameworks
      • Sonar Cloud is managed services, Sonar Qube is what JPL uses.
      • Keep with nomenclature of inner source and outer source
        • current consolidated list allows users to make their own decision.
      • JPL also uses tool Scrub, easy UI, report generation. includes code-QL.

    • Python Starter Kit: Trusted publishing and documentation refresh #167 (John Engelke)

      • John at testing phase before ready for potential infusion projects can try.
      • should CAE be involve/inform of this?
      • action item: @Paul/@Hook to reach out to CAE, Alex Decharez?
      • CAE also has existing guides that overlaps with SLIM
      • isn't CAE already like the implementing body?
      • SLIM can help funnel users to CAE tools.

    • #slim-users public-facing slack channel (Paul Ramirez)

      • NASA AMMOS slack org
        • has user channel that is linked to channel #nasa-ammos-aerie-users in JPL slack org
        • #mmgis-users
        • would SLIM like something similar
        • in conversations with NASA HQ if AMMOS slack org can take on public slack accounts...
      • how to we engage broader NASA community
      • upcoming use case: MAAP has its own slack org that we can also link with for engagement.
      • NASA slack org has more requirements for public users to join
      • Paul suggests to specifically link AMMOS slack org and JPL slack org. Implies new users will need to join AMMOS to be linked to JPL Slack slim channels e.g. #slim-users Projects with existing Slack orgs can join AMMOS via Slack Connect.

September 18th, 2024

Agenda

  • 10m News & Activity
    • @hookhua filling in leadership while @rishiverma out
    • Galen on developing a GenAI best practice
      • evaluation of different LLM models, best practices for code development,
      • one gap we experience is best practices for a code repo for how to structure utilization of LLM such that we can switch out different models when newer models are released. e.g. OpenAI o1 vs 4o as a seamless change.
      • handling large diversity of models and constant flux
      • best practices use of LLM for code gen.
      • we should put this into a ticket. assign to @galen (todo)
      • @hookhua: opportunity for adding best practices on prompt engineering for thinking step-by-step, agentic step and reasoning, reflectoin, and safety alignment. But many newer models like OpenAI o1 does this built-in. So what are some recommendations by model types?
    • @John has task working on user support tool by plugging into hugging-face.
      • will add ticket on best practices for ai user chatbot. how to structure LLM-based chatbot code for user support documentation. (todo)
      • what are costs associated with this? citizen science non-profit open source project.
    • access to GPUs for development?
      • JPL HPC program has some H100 GPUs with 64GB memory supporting 70B parameter llama models
      • ITSD Azure OpenAI API CoPilot pilot project - free for now, there is a possibility that they may start charging in FY25
    • 🔌 Infusion updates we're aware of
      • SPHEREx: will be following up. internal code repos at Caltech.
      • Unity: updated leaderboard, pull request of testing plan for all repositories, slim-cli readme update on infusion to multiple repositories
        • many not yet updated with: GitHub: Vulnerability Alerts GitHub: Code Scanning Alerts GitHub: Secret Scanning Alerts Secrets Detection Governance Model Continuous Testing Plan
      • slim-cli infusion to any other projects?
        • @ks trying it with ROSA (robot operating service agent?) repos and infusing best practices.
          • what is the state? any feedback? will follow up. encourage to use github discussions to foster community (todo)
      • potential new infusions
        • MAAP
          • platform code repos
          • UWG code repos
            • some of the code is in gitlab and some in github.
              • open ticket for testing on gitlab (todo @ks)
          • collaboration opportunities with NISAR?
            • many of this community code is in gitlab
        • Routing Application for Parallel computatIon of Discharge (RAPID)
          • https://github.com/c-h-david/rapid/
          • SLIM best practices open science from the start, different from existing SLIM software developer-centric repos.
          • potential for best practices for:
            • open science community engagement
            • derivative work
            • attribution
            • citations of published works, code, model, and datasets.
        • Zenodo
          • PDS also uses Zenodo for publishing artifacts and DOI citations.
          • on ImgSPEC/SISTER, we worked ORNL DAAC and LP DAAC. DAACs handled Zenodo.
      • Any other updates?
      • what is SLIM's readiness for projects with existing gitlab repos?
        • should SLIM CLI be less focused on github? graphql on github may not be as portable to other repos like gitlab and bitbucket.
  • 15m suggestions from community:
    • Cookiecutter (Lewis McGibbney)
    • slim-starterkit-python template for JPL GHE innersource projects? (Godwin Shen)
  • 10m Product & Docs Website
  • 10m Topic: FY25 Q1 plan - continuous delivery
  • SLIM casual lunch time after the community meeting

September 5th, 2024

Agenda

  • 10m News & Activity
    • 🔔 Reminder: @riverma fall plans and @hookhua leadership
    • GitHub co-pilot license conversations ongoing
      • Galen is working on developing a GenAI best practice
    • 🔌 Infusion updates we're aware of
      • OPERA: product website (PR: https://github.com/NASA-AMMOS/slim/pull/169) infused
      • Mission control systems integration, test & development group discussions for SLIM infusion (Gus Razo's group)
      • SPHEREx: Caltech project (Rishi will follow up and connect them with the SLIM team)
      • Any other updates?
  • 15m Topic: SLIM-CLI updates
    • Discuss infusion goals: IDS and Unity (KS will follow up and infuse slim-cli by end of September)
    • Feedback / discussion notes:
      • Paul suggested integrating complete information, including slim leaderboard and slim-cli.
    • slim-cli improvement & bug fixes: Pypi package by @nutjob4life (pip install slim-cli), unit tests, forking/custom-remote support, prompt improvement by @riverma, best practice link fix, proper error message & exit when OpenAI/Azure key missing by @yunks128
    • current issues: https://github.com/NASA-AMMOS/slim-cli/issues
  • 15m Next SLIM release:
  • 10m Product & Docs Website
  • 10m Topic: FY25 Q1 plan - continuous delivery
  • SLIM casual lunch time after the community meeting

August 21st, 2024

Agenda

  • 10m News & Activity
    • 🔔 Reminder: @riverma fall plans and @hookhua leadership
    • NASA GenAI Workshop (Monday, August 19th): SLIM presented at 10:55am - 11:25am
    • ICSE toolkit discussions for SLIM infusion
    • GitHub co-pilot license conversations ongoing
      • Possible SLIM guide?
    • 🔌 Infusion updates we're aware of
      • NISAR ADT: container scanning and secrets scanning SLIM best practices
      • ROSA: README, and other repo essentials
      • Any other updates?
      • Possible inroads with MAAP (@hookhua)
  • 20m Topic: SLIM-CLI demonstration (to be discussed at GenAI workshop)
    • Demo: @yunks128: slim-cli apply --use-ai
    • Feedback / discussion notes:
      • Hook: This could be used to upgrade best practices with new versions
      • Rishi: we should be smart about the context we send per best practice. For example, send GitHub API info instead of code repo information for Governance guide, etc.
  • 15m Planning board discussion for FY25
  • 10m Changelog guidance
  • 5m Open Pull Requests:

August 8th, 2024

Agenda

  • 10m News & Activity
    • 🔔 SpaceOps paper: abstract submitted: Rishi, KS, Paul
    • 🔔 Reminder: NASA GenAI Workshop (Monday, August 19th): SLIM will present at 10:55am - 11:25am
    • 🔔 Reminder: @riverma fall plans
    • 🔌 Infusion updates we're aware of
      • OPERA SDS: grype (container scanning guide) infused, secrets detection guide infused, contributing guide currently under infusion
      • Unity SDS: continuous testing (unity-sps), README guides (unity-cs)
      • Any other updates?
        • IDS
        • PDS-EN
        • HySDS
        • F'Prime
      • Possible inroads with ICSE toolkit
      • ExoPlanet: possible inroads of infusion
      • SPHEREx: possible inroads of infusion
  • 20m Topic: SLIM-CLI demonstration (to be discussed at GenAI workshop)
    • Feedback on tool?
    • Demo: @riverma: gap identification using slim leaderboard scans
    • Demo: @riverma: slim-cli list, apply, apply-deploy
    • Demo: @yunks128: slim-cli apply --use-ai
    • Feedback / discussion notes:
      • Adrian: tool seems git-centric. Is it architected in an extensible way?
      • Hook: should we team up with the institution to infuse best practices with many projects?
      • Sean: can we put this on PyPI?
      • John: GraphQL can be abstracted
  • 10m Changelog guidance
  • 5m Open Pull Requests:

July 23rd, 2024

Agenda

  • 10m News & Activity
    • 🔔 SpaceOps paper
    • 🔔 NASA GenAI Workshop
    • 🔔 @riverma fall plans
    • 🔔 Possible new community members
      • (@jamesray): Launchbox + others
      • SPHEREx science data system
    • 🔌 Infusion updates we're aware of
      • OPERA SDS container scanning
      • PDS EN and container scanning
      • Possible inroads to 'Common Software & Solutions' team
      • Possible inroads to flight electronics testing
      • Possible contributions from CMU
  • 20m Topic: Contractual / Requirements-Based SLIM Infusion
    • Demo (@riverma): latest leaderboard reports. https://github.com/nasa-ammos/slim-leaderboard
    • @Hook and @Galen mentioned how SLIM's "best practices" may not be infused as much if projects are not required to. Discuss the options, pros, costs of requirements based infusion.
    • Discussion notes:
      • It may be an up-hill battle
      • Possibly software management plans (not explicitly calling out SLIM but calling out artifact needs)
      • Idea for carrots: awards for passing SLIM checks well?
      • Link to existing standards or req's within leaderboard
      • Example L4 or L5 reqs
  • 15m Topic: SLIM-CLI
    • @riverma: Hackathon outcome and next steps
    • @yunks128: AI features and progress
  • 5m Container Security
    • @riverma OPERA SDS, PDS-EN currently trialing this PR - others?
    • Possible talk on this upcoming
  • 5m DevOps Best Practice Guide @riverma
    • With the departure of @ddalton, @riverma to pick up tab temporarily and seek contributions from sagar j. and community.
    • Projects that are doing DevOps / continuous automation well - interested in collaborating on this?

July 11th, 2024

Agenda

  • 10m News & Activity
    • 🔔 New meeting series merging previously separated steering committee and developer meetings. Poll for new time among currently community members to be sent out.
    • 🔔 New Continuous Testing guide on SLIM! (thank you @yunks128). See new v1.5 Release Notes (volunteers to help disseminate?)
      • Discussion:
        • @Stirling: shall we automate this?
        • @Stirling: Slack workspace for SLIM
        • @Hook: having a mailing list allows for non-JPLers to receive updates (+1'd)
        • @Sean: Discord, RSS, WebHooks, etc. are other options
        • @Hook: moving comms to an open source setting early is a better idea to reduce friction
      • Action Items:
        • Volunteers for future sharing of SLIM releases
          • unity-sds (Galen)
          • group (Galen, Adrian)
          • mighty-devs (Paul)
          • hysds-community (Hook)
    • 🔔 New SLIM community members: PDS-EN (POC's: Sean K., Jordan P.)
    • 🔌 Infusion updates we're aware of
      • Unity SDS (SPS) trialing Continuous Testing Guide in this PR
        • Action Items:
          • @riverma to coordinate with @Galen on this PR
          • @riverma to include a topic for next week for infusion via contractual / requirements
      • OPERA SDS successfully trialed Secrets Detection guide. Working on infusing automation recommendations.
      • HySDS currently trialing Contributing Guide in this ticket
    • 🅿️ Recently updated pull requests
      • Discussion
        • @Hook: can we use LLM's to generate draft release notes that summarize the developer updates?
        • @Paul: co-pilot maybe does this
    • ℹ️ Recently updated issues
  • 20m SLIM-CLI @riverma @yunks128
    • @riverma overview of ticket and latest plan
    • @yunks128 first draft of the use_ai function
    • Interested beta testers?
    • Discussion:
      • @Galen @Adrian: perhaps we should rebrand the "--use-ai" to language that indicates it "just works"
Clone this wiki locally