-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Provide a historical view of all sessions including session statistics in the DB console and SQL CLI #67888
Labels
A-sql-cli-observability
Issues related to surfacing SQL observability in SHOW, CRDB_INTERNAL, SYSTEM, etc.
A-sql-console-general
SQL Observability issues on the DB console spanning multiple areas. Includes Cockroach Cloud Console
C-enhancement
Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
Comments
kevin-v-ngo
added
C-enhancement
Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
T-sql-observability
A-sql-console-general
SQL Observability issues on the DB console spanning multiple areas. Includes Cockroach Cloud Console
A-sql-cli-observability
Issues related to surfacing SQL observability in SHOW, CRDB_INTERNAL, SYSTEM, etc.
labels
Jul 22, 2021
Would this view want to expire sessions that are closed, or keep them around? This won't be easily possible as written - it'll be too much data to store. We'll probably need to think of some sampling strategy or another way to avoid keeping all of the fingerprints in memory per session. |
gtr
pushed a commit
to gtr/cockroach
that referenced
this issue
Mar 9, 2022
This the first phase of [cockroachdb#67888](cockroachdb#67888) which simply adds a FIFO cache for closed sessions. The cache's eviction policy is both row-based and time-based. Both variables can be set when initialzing the cache, as `capacity` and `timeToLive`, respectively. Further PRs will focus on exposing these closed sessions to the frontend via the `ListSessions` endpoint, optimizing the capacity and timeToLive variables, and making appropriate UI changes. Release justification: Providing a historical view of sessions. Release note: None
gtr
pushed a commit
to gtr/cockroach
that referenced
this issue
Mar 12, 2022
This the first phase of cockroachdb#67888 which simply adds a FIFO cache for closed sessions. The cache's eviction policy is both row-based and time-based. Both the capacity and the time to live can be set using the `settings` package. Further PRs will focus on exposing these closed sessions to the frontend via the `ListSessions` endpoint, optimizing the capacity and timeToLive variables, and making appropriate UI changes. Release justification: low risk high benefit change Release note: None
gtr
pushed a commit
to gtr/cockroach
that referenced
this issue
Mar 12, 2022
This the first phase of cockroachdb#67888 which simply adds a FIFO cache for closed sessions. The cache's eviction policy is both row-based and time-based. Both the capacity and the time to live can be set using the `settings` package. Further PRs will focus on exposing these closed sessions to the frontend via the `ListSessions` endpoint, optimizing the capacity and timeToLive variables, and making appropriate UI changes. Release justification: low risk high benefit change Release note: None
gtr
pushed a commit
to gtr/cockroach
that referenced
this issue
Mar 22, 2022
This the first phase of cockroachdb#67888 which simply adds a FIFO cache for closed sessions. The cache's eviction policy is both row-based and time-based. Both the capacity and the time to live can be set using the `settings` package. Further PRs will focus on exposing these closed sessions to the frontend via the `ListSessions` endpoint, optimizing the capacity and timeToLive variables, and making appropriate UI changes. Release justification: low risk high benefit change Release note: None
gtr
pushed a commit
to gtr/cockroach
that referenced
this issue
Mar 22, 2022
This the first phase of cockroachdb#67888 which simply adds a FIFO cache for closed sessions. The cache's eviction policy is both row-based and time-based. Both the capacity and the time to live can be set using the `settings` package. Further PRs will focus on exposing these closed sessions to the frontend via the `ListSessions` endpoint, optimizing the capacity and timeToLive variables, and making appropriate UI changes. Release justification: low risk high benefit change Release note: None
gtr
pushed a commit
to gtr/cockroach
that referenced
this issue
Mar 23, 2022
This the first phase of cockroachdb#67888 which simply adds a FIFO cache for closed sessions. The cache's eviction policy is both row-based and time-based. Both the capacity and the time to live can be set using the `settings` package. Further PRs will focus on exposing these closed sessions to the frontend via the `ListSessions` endpoint, optimizing the capacity and timeToLive variables, and making appropriate UI changes. Release justification: low risk high benefit change Release note: None
gtr
pushed a commit
to gtr/cockroach
that referenced
this issue
Mar 23, 2022
This the first phase of cockroachdb#67888 which simply adds a FIFO cache for closed sessions. The cache's eviction policy is both row-based and time-based. Both the capacity and the time to live can be set using the `settings` package. Further PRs will focus on exposing these closed sessions to the frontend via the `ListSessions` endpoint, optimizing the capacity and timeToLive variables, and making appropriate UI changes. Release justification: low risk high benefit change Release note: None
gtr
pushed a commit
to gtr/cockroach
that referenced
this issue
Mar 23, 2022
This the first phase of cockroachdb#67888 which simply adds a FIFO cache for closed sessions. The cache's eviction policy is both row-based and time-based. Both the capacity and the time to live can be set using the `settings` package. Further PRs will focus on exposing these closed sessions to the frontend via the `ListSessions` endpoint, optimizing the capacity and timeToLive variables, and making appropriate UI changes. Release note: None
gtr
pushed a commit
to gtr/cockroach
that referenced
this issue
Mar 23, 2022
This the first phase of cockroachdb#67888 which simply adds a FIFO cache for closed sessions. The cache's eviction policy is both row-based and time-based. Both the capacity and the time to live can be set using the `settings` package. Further PRs will focus on exposing these closed sessions to the frontend via the `ListSessions` endpoint, optimizing the capacity and timeToLive variables, and making appropriate UI changes. Release note: None
gtr
pushed a commit
to gtr/cockroach
that referenced
this issue
Mar 24, 2022
This the first phase of cockroachdb#67888 which simply adds a FIFO cache for closed sessions. The cache's eviction policy is both row-based and time-based. Both the capacity and the time to live can be set using the `settings` package. Further PRs will focus on exposing these closed sessions to the frontend via the `ListSessions` endpoint, optimizing the capacity and timeToLive variables, and making appropriate UI changes. Release note: None
gtr
pushed a commit
to gtr/cockroach
that referenced
this issue
Mar 24, 2022
This the first phase of cockroachdb#67888 which simply adds a FIFO cache for closed sessions. The cache's eviction policy is both row-based and time-based. Both the capacity and the time to live can be set using the `settings` package. Further PRs will focus on exposing these closed sessions to the frontend via the `ListSessions` endpoint, optimizing the capacity and timeToLive variables, and making appropriate UI changes. Release note: None
gtr
pushed a commit
to gtr/cockroach
that referenced
this issue
Mar 24, 2022
This the first phase of cockroachdb#67888 which simply adds a FIFO cache for closed sessions. The cache's eviction policy is both row-based and time-based. Both the capacity and the time to live can be set using the `settings` package. Further PRs will focus on exposing these closed sessions to the frontend via the `ListSessions` endpoint, optimizing the capacity and timeToLive variables, and making appropriate UI changes. Release note: None
craig bot
pushed a commit
that referenced
this issue
Mar 25, 2022
77577: sql: add in-memory sessions FIFO cache for closed sessions. r=gtr a=gtr This the first phase of #67888 which simply adds a FIFO cache for closed sessions. The cache's eviction policy is both row-based and time-based. Both the capacity and the time to live can be set using the `settings` package. Further PRs will focus on exposing these closed sessions to the frontend via the `ListSessions` endpoint, optimizing the capacity and timeToLive variables, and making appropriate UI changes. Release note: None Co-authored-by: Gerardo Torres <[email protected]>
gtr
pushed a commit
that referenced
this issue
Mar 28, 2022
This is the second phase of #67888 which is to expose closed sessions to the ListSessions endpoint. Previously, the ListSessions endpoint only returned open sessions. This change builds on top of the previous PR to add the `ClosedSessionsCache` and now allows the ListSessions endpoint to also return closed sessions in its response. Additionally, the `serverpb.Session` object was updated to inlcude new `end` and `status` fields which specify the time the session ended and the status (opened, closed) of the session, respectively. Release note (api change): `ListSessions` now returns closed sessions in addition to open sessions; `serverpb.Session` now has `end` and `status` fields.
gtr
pushed a commit
to gtr/cockroach
that referenced
this issue
Apr 5, 2022
This is the second phase of cockroachdb#67888 which is to expose closed sessions to the ListSessions endpoint. Previously, the ListSessions endpoint only returned open sessions. This change builds on top of the previous PR to add the `ClosedSessionsCache` and now allows the ListSessions endpoint to also return closed sessions in its response. Additionally, the `serverpb.Session` object was updated to inlcude new `end` and `status` fields which specify the time the session ended and the status (opened, closed) of the session, respectively. Release note (api change): `ListSessions` now returns closed sessions in addition to open sessions; `serverpb.Session` now has `end` and `status` fields.
gtr
pushed a commit
that referenced
this issue
Apr 14, 2022
This is the second phase of #67888 which is to expose closed sessions to the ListSessions endpoint. Previously, the ListSessions endpoint only returned open sessions. This change builds on top of the previous PR to add the `ClosedSessionsCache` and now allows the ListSessions endpoint to also return closed sessions in its response. The `ListSessionsRequest` object was edited to include a flag `exclude_closed_sessions` which is a boolean to exclude closed sessions. If unspecified, defaults to false and closed sessions are included in the response. Additionally, the `serverpb.Session` object was updated to include new `end` and `status` fields which specify the time the session ended and the status (opened, closed) of the session, respectively. Release note (api change): `ListSessions` now returns closed sessions in addition to open sessions; `ListSessionsRequest` now has a `exclude_closed_sessions` flag; `serverpb.Session` now has `end` and `status` fields.
gtr
pushed a commit
that referenced
this issue
Apr 14, 2022
This is the second phase of #67888 which is to expose closed sessions to the ListSessions endpoint. Previously, the ListSessions endpoint only returned open sessions. This change builds on top of the previous PR to add the `ClosedSessionsCache` and now allows the ListSessions endpoint to also return closed sessions in its response. The `ListSessionsRequest` object was edited to include a flag `exclude_closed_sessions` which is a boolean to exclude closed sessions. If unspecified, defaults to false and closed sessions are included in the response. Additionally, the `serverpb.Session` object was updated to include new `end` and `status` fields which specify the time the session ended and the status (opened, closed) of the session, respectively. Release note (api change): `ListSessions` now returns closed sessions in addition to open sessions; `ListSessionsRequest` now has a `exclude_closed_sessions` flag; `serverpb.Session` now has `end` and `status` fields.
gtr
pushed a commit
that referenced
this issue
Apr 18, 2022
This is the second phase of #67888 which is to expose closed sessions to the ListSessions endpoint. Previously, the ListSessions endpoint only returned open sessions. This change builds on top of the previous PR to add the `ClosedSessionsCache` and now allows the ListSessions endpoint to also return closed sessions in its response. The `ListSessionsRequest` object was edited to include a flag `exclude_closed_sessions` which is a boolean to exclude closed sessions. If unspecified, defaults to false and closed sessions are included in the response. Additionally, the `serverpb.Session` object was updated to include new `end` and `status` fields which specify the time the session ended and the status (active, idle, closed) of the session, respectively. Release note (api change): `ListSessions` now returns closed sessions in addition to open sessions; `ListSessionsRequest` now has a `exclude_closed_sessions` flag; `serverpb.Session` now has `end` and `status` fields.
gtr
pushed a commit
that referenced
this issue
Apr 19, 2022
This is the second phase of #67888 which is to expose closed sessions to the ListSessions endpoint. Previously, the ListSessions endpoint only returned open sessions. This change builds on top of the previous PR to add the `ClosedSessionsCache` and now allows the ListSessions endpoint to also return closed sessions in its response. The `ListSessionsRequest` object was edited to include a flag `exclude_closed_sessions` which is a boolean to exclude closed sessions. If unspecified, defaults to false and closed sessions are included in the response. Additionally, the `serverpb.Session` object was updated to include new `end` and `status` fields which specify the time the session ended and the status (active, idle, closed) of the session, respectively. Release note (api change): `ListSessions` now returns closed sessions in addition to open sessions; `ListSessionsRequest` now has a `exclude_closed_sessions` flag; `serverpb.Session` now has `end` and `status` fields.
gtr
pushed a commit
that referenced
this issue
Apr 20, 2022
This is the second phase of #67888 which is to expose closed sessions to the ListSessions endpoint. Previously, the ListSessions endpoint only returned open sessions. This change builds on top of the previous PR to add the `ClosedSessionsCache` and now allows the ListSessions endpoint to also return closed sessions in its response. The `ListSessionsRequest` object was edited to include a flag `exclude_closed_sessions` which is a boolean to exclude closed sessions. If unspecified, defaults to false and closed sessions are included in the response. Additionally, the `serverpb.Session` object was updated to include new `end` and `status` fields which specify the time the session ended and the status (active, idle, closed) of the session, respectively. Release note (api change): `ListSessions` now returns closed sessions in addition to open sessions; `ListSessionsRequest` now has a `exclude_closed_sessions` flag; `serverpb.Session` now has `end` and `status` fields.
gtr
pushed a commit
that referenced
this issue
Apr 20, 2022
Fixes #67888, #79914. Previously, the sessions page UI did not support displaying closed sessions and did not support the ability to filter by username or session status. This commit adds the "Closed" session status to closed sessions and adds the ability to filter by username and session status. Release note (ui change): sessions overview and session details pages now display closed sessions; sessions overview page now has username and session status filters.
gtr
pushed a commit
that referenced
this issue
Apr 21, 2022
This is the second phase of #67888 which is to expose closed sessions to the ListSessions endpoint. Previously, the ListSessions endpoint only returned open sessions. This change builds on top of the previous PR to add the `ClosedSessionsCache` and now allows the ListSessions endpoint to also return closed sessions in its response. The `ListSessionsRequest` object was edited to include a flag `exclude_closed_sessions` which is a boolean to exclude closed sessions. If unspecified, defaults to false and closed sessions are included in the response. Additionally, the `serverpb.Session` object was updated to include new `end` and `status` fields which specify the time the session ended and the status (active, idle, closed) of the session, respectively. Release note (api change): `ListSessions` now returns closed sessions in addition to open sessions; `ListSessionsRequest` now has a `exclude_closed_sessions` flag; `serverpb.Session` now has `end` and `status` fields.
gtr
pushed a commit
that referenced
this issue
Apr 21, 2022
This is the second phase of #67888 which is to expose closed sessions to the ListSessions endpoint. Previously, the ListSessions endpoint only returned open sessions. This change builds on top of the previous PR to add the `ClosedSessionsCache` and now allows the ListSessions endpoint to also return closed sessions in its response. The `ListSessionsRequest` object was edited to include a flag `exclude_closed_sessions` which is a boolean to exclude closed sessions. If unspecified, defaults to false and closed sessions are included in the response. Additionally, the `serverpb.Session` object was updated to include new `end` and `status` fields which specify the time the session ended and the status (active, idle, closed) of the session, respectively. Release note (api change): `ListSessions` now returns closed sessions in addition to open sessions; `ListSessionsRequest` now has a `exclude_closed_sessions` flag; `serverpb.Session` now has `end` and `status` fields.
gtr
pushed a commit
that referenced
this issue
Apr 21, 2022
This is the second phase of #67888 which is to expose closed sessions to the ListSessions endpoint. Previously, the ListSessions endpoint only returned open sessions. This change builds on top of the previous PR to add the `ClosedSessionsCache` and now allows the ListSessions endpoint to also return closed sessions in its response. The `ListSessionsRequest` object was edited to include a flag `exclude_closed_sessions` which is a boolean to exclude closed sessions. If unspecified, defaults to false and closed sessions are included in the response. Additionally, the `serverpb.Session` object was updated to include new `end` and `status` fields which specify the time the session ended and the status (active, idle, closed) of the session, respectively. Release note (api change): `ListSessions` now returns closed sessions in addition to open sessions; `ListSessionsRequest` now has a `exclude_closed_sessions` flag; `serverpb.Session` now has `end` and `status` fields.
gtr
pushed a commit
that referenced
this issue
Apr 21, 2022
This is the second phase of #67888 which is to expose closed sessions to the ListSessions endpoint. Previously, the ListSessions endpoint only returned open sessions. This change builds on top of the previous PR to add the `ClosedSessionsCache` and now allows the ListSessions endpoint to also return closed sessions in its response. The `ListSessionsRequest` object was edited to include a flag `exclude_closed_sessions` which is a boolean to exclude closed sessions. If unspecified, defaults to false and closed sessions are included in the response. Additionally, the `serverpb.Session` object was updated to include new `end` and `status` fields which specify the time the session ended and the status (active, idle, closed) of the session, respectively. Release note (api change): `ListSessions` now returns closed sessions in addition to open sessions; `ListSessionsRequest` now has a `exclude_closed_sessions` flag; `serverpb.Session` now has `end` and `status` fields.
gtr
pushed a commit
that referenced
this issue
Apr 21, 2022
This is the second phase of #67888 which is to expose closed sessions to the ListSessions endpoint. Previously, the ListSessions endpoint only returned open sessions. This change builds on top of the previous PR to add the `ClosedSessionsCache` and now allows the ListSessions endpoint to also return closed sessions in its response. The `ListSessionsRequest` object was edited to include a flag `exclude_closed_sessions` which is a boolean to exclude closed sessions. If unspecified, defaults to false and closed sessions are included in the response. Additionally, the `serverpb.Session` object was updated to include new `end` and `status` fields which specify the time the session ended and the status (active, idle, closed) of the session, respectively. Release note (api change): `ListSessions` now returns closed sessions in addition to open sessions; `ListSessionsRequest` now has a `exclude_closed_sessions` flag; `serverpb.Session` now has `end` and `status` fields.
craig bot
pushed a commit
that referenced
this issue
Apr 22, 2022
78650: server, sql: expose closed sessions to ListSessions endpoint r=gtr a=gtr This is the second phase of #67888 which is to expose closed sessions to the ListSessions endpoint. Previously, the ListSessions endpoint only returned open sessions. This change builds on top of the previous PR to add the `ClosedSessionsCache` and now allows the ListSessions endpoint to also return closed sessions in its response. The `ListSessionsRequest` object was edited to include a flag `exclude_closed_sessions` which is a boolean to exclude closed sessions. If unspecified, defaults to false and closed sessions are included in the response. Additionally, the `serverpb.Session` object was updated to include new `end` and `status` fields which specify the time the session ended and the status (opened, closed) of the session, respectively. To test the changes, I made a test `TestListClosedSessions` which creates three users. For each user, we create 10 closed sessions, 5 open sessions, and 3 idle sessions. - The closed sessions are created by opening 10 DB connections and then closing them right after: ```go // Open 10 sessions for each user and then close them. for _, user := range users { for i := 0; i < 10; i++ { targetDB := getUserConn(t, user, testCluster.Server(0)) dbs = append(dbs, targetDB) sqlutils.MakeSQLRunner(targetDB).Exec(t, `SELECT version()`) } } for _, db := range dbs { err := db.Close() require.NoError(t, err) } ``` - The open sessions are created by opening up 5 DB connections and running `pg_sleep(30)` concurrently: ```go // Open 5 sessions for the user and leave them open by running pg_sleep(30). for _, user := range users { for i := 0; i < 5; i++ { wg.Add(1) go func(user string) { // Open a session for the target user. targetDB := getUserConn(t, user, testCluster.Server(0)) defer targetDB.Close() defer wg.Done() sqlutils.MakeSQLRunner(targetDB).Exec(t, `SELECT pg_sleep(30)`) }(user) } } ``` - The idle sessions are created by opening up 3 DB connections and running `SELECT version()` (which finishes executing almost instantaneously) but deferring the `db.Close()` call until the end of the test: ```go // Open 3 sessions for each user and leave them idle by running version(). for _, user := range users { for i := 0; i < 3; i++ { targetDB := getUserConn(t, user, testCluster.Server(0)) defer targetDB.Close() sqlutils.MakeSQLRunner(targetDB).Exec(t, `SELECT version()`) } } ``` These are the results for the three users: ``` username: test_user_a ------------------------------------- Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: ACTIVE Status: ACTIVE Status: ACTIVE Status: ACTIVE Status: ACTIVE Status: IDLE Status: IDLE Status: IDLE username: test_user_b ------------------------------------- Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: ACTIVE Status: ACTIVE Status: ACTIVE Status: ACTIVE Status: ACTIVE Status: IDLE Status: IDLE Status: IDLE username: test_user_c ------------------------------------- Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: CLOSED Status: ACTIVE Status: ACTIVE Status: ACTIVE Status: ACTIVE Status: ACTIVE Status: IDLE Status: IDLE Status: IDLE ``` Release note (api change): `ListSessions` now returns closed sessions in addition to open sessions; `ListSessionsRequest` now has a `exclude_closed_sessions` flag; `serverpb.Session` now has `end` and `status` fields. Co-authored-by: Gerardo Torres <[email protected]>
gtr
pushed a commit
to gtr/cockroach
that referenced
this issue
Apr 22, 2022
Fixes cockroachdb#67888, cockroachdb#79914. Previously, the sessions page UI did not support displaying closed sessions and did not support the ability to filter by username or session status. This commit adds the "Closed" session status to closed sessions and adds the ability to filter by username and session status. Release note (ui change): sessions overview and session details pages now display closed sessions; sessions overview page now has username and session status filters
gtr
pushed a commit
to gtr/cockroach
that referenced
this issue
Apr 25, 2022
Fixes cockroachdb#67888, cockroachdb#79914. Previously, the sessions page UI did not support displaying closed sessions and did not support the ability to filter by username or session status. This commit adds the "Closed" session status to closed sessions and adds the ability to filter by username and session status. Release note (ui change): sessions overview and session details pages now display closed sessions; sessions overview page now has username and session status filters
gtr
pushed a commit
to gtr/cockroach
that referenced
this issue
Apr 26, 2022
Fixes cockroachdb#67888, cockroachdb#79914. Previously, the sessions page UI did not support displaying closed sessions and did not support the ability to filter by username or session status. This commit adds the "Closed" session status to closed sessions and adds the ability to filter by username and session status. Release note (ui change): sessions overview and session details pages now display closed sessions; sessions overview page now has username and session status filters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
A-sql-cli-observability
Issues related to surfacing SQL observability in SHOW, CRDB_INTERNAL, SYSTEM, etc.
A-sql-console-general
SQL Observability issues on the DB console spanning multiple areas. Includes Cockroach Cloud Console
C-enhancement
Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
We should provide a historical view of Sessions as we have with Transactions and Statements. This issue tracks the V1 experience of persisting "Closed" Sessions for a period of time. V2 builds on top of this by adding additional session details and statistics.
Adding historical sessions including additional information would provide a higher-level grouping of customer workloads at the session-level. CockroachDB workloads typically issue single statement transactions (implicit) and allowing a grouping at the session level will allow users to:
For this issue, crdb_internal.cluster_session and the existing console information is captured and surfaced for closed sessions. New information to add includes the "Closed" status and End time.
Given the potential size of data there are a few options we can explore in terms of the retention period:
Link to the current Figma
Epic: CRDB-9087
Jira issue: CRDB-8763
The text was updated successfully, but these errors were encountered: