Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipeline statistics #156

Open
timonv opened this issue Jul 12, 2024 · 0 comments
Open

Pipeline statistics #156

timonv opened this issue Jul 12, 2024 · 0 comments
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed

Comments

@timonv
Copy link
Member

timonv commented Jul 12, 2024

Is your feature request related to a problem? Please describe.
I would like to know what my pipeline costs and have data to effectively improve it.

Describe the solution you'd like
After a pipeline completes, I should have access to numbers of nodes, errors, usage per model/llm, perhaps more. Would be cool if part of this data is also available via tracing.

Some musings I had on this:

  • Separate aggregation thread or atomics?
  • Would be nice if we can get the data while its running (apps can then display progress)
  • There might be a way to estimate the total amount of nodes at any stage (depending on the type of stream)
  • Also nice if the same data can be used to enhance tracing, preferably matching the beta otel llm spec
@timonv timonv added enhancement New feature or request good first issue Good for newcomers labels Jul 12, 2024 — with Connector GitHub for Miro
@timonv timonv added the help wanted Extra attention is needed label Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant