Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update docs and metadata for package:sdk_triage_bot #266

Merged
merged 2 commits into from
Jun 7, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/ISSUE_TEMPLATE/sdk_triage_bot.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
name: "package:sdk_triage_bot"
about: "Create a bug or file a feature request against package:sdk_triage_bot."
labels: "package:sdk_triage_bot"
---
4 changes: 4 additions & 0 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,7 @@
'package:repo_manage':
- changed-files:
- any-glob-to-any-file: 'pkgs/repo_manage/**/*'

'package:sdk_triage_bot':
- changed-files:
- any-glob-to-any-file: 'pkgs/sdk_triage_bot/**/*'
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ This repository is home to general Dart Ecosystem tools and packages.
| [dart_flutter_team_lints](pkgs/dart_flutter_team_lints/) | An analysis rule set used by the Dart and Flutter teams. | [![pub package](https://img.shields.io/pub/v/dart_flutter_team_lints.svg)](https://pub.dev/packages/dart_flutter_team_lints) |
| [firehose](pkgs/firehose/) | A tool to automate publishing of Pub packages from GitHub actions. | [![pub package](https://img.shields.io/pub/v/firehose.svg)](https://pub.dev/packages/firehose) |
| [repo_manage](pkgs/repo_manage/) | Miscellaneous issue, repo, and PR query tools. | |
| [sdk_triage_bot](pkgs/sdk_triage_bot/) | A triage automation tool for dart-lang/sdk issues. | |

## Publishing automation

Expand Down
23 changes: 20 additions & 3 deletions pkgs/sdk_triage_bot/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,29 @@ human would. This includes:

## Bot trigger and entry-point

TODO: doc
This bot is generally triggered by a GitHub workflow listening for new issues
on the dart-lang/sdk repo.

See https://github.com/dart-lang/sdk/blob/main/.github/workflows/issue-triage.yml.

## Overview

TODO: doc
The general workflow of the tool is:

- download the issue information (existing labels, title, first comment)
- ask Gemini to summarize the issue (see [prompts](lib/src/prompts.dart))
- ask Gemini to classify the issue (see [prompts](lib/src/prompts.dart))
- create a comment on the issue ( `@dart-github-bot`) with the summary;
apply any labels produced as part of the classification

## Tuning

TODO: doc
We create a tuned Gemini model in order to improve the performance of
classification. This involves:

- downloading recent, already triaged issues (~800 issues)
- writing them in a format suitable for tuning (either .csv or .jsonl)
- tuning via the Gemini APIs; this gives us a new model name to use when
calling Gemini (e.g., `model: 'tunedModels/sdk-triage-tuned-prompt-1l96e2n'`)

See [tool/create_tuning_data.dart](tool/create_tuning_data.dart).