Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send platform usage statistics to users #3161

Open
mzueva opened this issue Mar 29, 2023 · 4 comments
Open

Send platform usage statistics to users #3161

mzueva opened this issue Mar 29, 2023 · 4 comments
Assignees
Labels
kind/enhancement New feature or request

Comments

@mzueva
Copy link
Collaborator

mzueva commented Mar 29, 2023

Background
It would be useful to send to users a letter on scheduled basis with insights on the platform usage.

Approach
Let's create a python script which generates letters with user's stats for the specified period. Script arguments: from (date), to (date), list of users (reports shall be generated for all specified users), list of roles (reports shall be generated for all users belonging to groups). Lists of users and roles may be specified simultaneously.

Report shall include the following data:

Overall:

  • Compute hours (overall): hours
  • Runs count: count
  • Storage usage
    • Read requests: count
    • Write requests: count
  • Usage weight: %% (user costs / platform costs for the same period)
  • Login time: hours

Most-used:

  • Compute type
    • CPU: hours
    • GPU: hours
  • Clusters
    • Compute: hours (total time master instances were running)
    • Worker nodes CPU / GPU: count (number of worker nodes started)
  • Top 3 instance types: count
  • Top 3 pipelines: count
  • Top 3 tools: count
  • Top 3 used buckets: count
  • Top 3 run capabilities: hours (total time of the runs with a specific capability)
@mzueva mzueva added the kind/enhancement New feature or request label Mar 29, 2023
mzueva added a commit that referenced this issue May 11, 2023
mzueva added a commit that referenced this issue May 11, 2023
* Issue 3161 Send platform usage statistics to users

* Issue 3161 Send platform usage statistics to users

* Issue 3161 Send platform usage statistics to users - fix remarks

* Issue 3161 Send platform usage statistics to users - add templates

* Issue #3161 PMD fix

---------

Co-authored-by: mzueva <[email protected]>
mzueva added a commit that referenced this issue May 17, 2023
mzueva added a commit that referenced this issue May 17, 2023
@maryvictol
Copy link
Collaborator

maryvictol commented May 24, 2023

@mzueva , @okolesn
Bug:
The report isn't generated for a list of users (separated by a comma or space).
Message in the log: No users found to collect and send statistics.

@maryvictol
Copy link
Collaborator

@mzueva , @okolesn
Bug:
Values Storage write requests count: 0. Storage read requests count: 0. are returned in reports for all users.

@maryvictol
Copy link
Collaborator

maryvictol commented May 24, 2023

@mzueva , @okolesn
Bug:
Method POST /pipeline/restapi/user-notification/message gives error
{"message":"Failed to evaluate expression 'hasRole('ADMIN') OR @notificationPermissionManager.hasPermission(#notification)'","status":"ERROR"}
for non-admin users.
As result non-admin user can't close the notification.

@maryvictol
Copy link
Collaborator

@mzueva , @okolesn Bug: Method POST /pipeline/restapi/user-notification/message gives error {"message":"Failed to evaluate expression 'hasRole('ADMIN') OR @notificationPermissionManager.hasPermission(#notification)'","status":"ERROR"} for non-admin users. As result non-admin user can't close the notification.

Verified as fixed

mzueva added a commit that referenced this issue Jul 25, 2023
* Issue 3161 Send platform usage statistics to users

* Issue 3161 Send platform usage statistics to users

* Issue 3161 Send platform usage statistics to users - fix remarks

* Issue 3161 Send platform usage statistics to users - add templates

* Issue #3161 PMD fix

---------

Co-authored-by: mzueva <[email protected]>
mzueva added a commit that referenced this issue Jul 25, 2023
mzueva added a commit that referenced this issue Jul 25, 2023
mzueva added a commit that referenced this issue Jul 25, 2023
mzueva added a commit that referenced this issue Jul 27, 2023
ekazachkova added a commit that referenced this issue Aug 22, 2023
ekazachkova added a commit that referenced this issue Aug 29, 2023
mzueva pushed a commit that referenced this issue Aug 29, 2023
* Issue #3161: Send platform usage statistics to users - respect pause run stated during run stat calculation

* Issue #3161: Send platform usage statistics to users - use billing data for run duration calculation
mzueva added a commit that referenced this issue Aug 30, 2023
mzueva added a commit that referenced this issue Aug 30, 2023
mzueva added a commit that referenced this issue Aug 30, 2023
sidoruka pushed a commit that referenced this issue Aug 31, 2023
(cherry picked from commit 5f9c763)
sidoruka pushed a commit that referenced this issue Aug 31, 2023
(cherry picked from commit 6478ac2)
mzueva added a commit that referenced this issue Jan 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants