Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

admission: include request priority and queue name in WorkQueue tracing #113990

Closed
sumeerbhola opened this issue Nov 7, 2023 · 3 comments
Closed
Assignees
Labels
A-admission-control branch-master Failures and bugs on the master branch. branch-release-23.2 Used to mark GA and release blockers, technical advisories, and bugs for 23.2 C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) GA-blocker T-admission-control Admission Control

Comments

@sumeerbhola
Copy link
Collaborator

sumeerbhola commented Nov 7, 2023

trace statements like

work kv deadline expired while waiting: deadline: 2023-10-14 10:20:57.098424558 +0000 UTC m=+377.360809249, start: 2023-10-14 10:19:57.098548 +0000 UTC m=+317.360932704, dur: 1m0.039413971s

don't tell us whether it was a store queue (and elastic or regular) or the CPU queue.

Jira issue: CRDB-33285

@sumeerbhola sumeerbhola added the C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) label Nov 7, 2023
@sumeerbhola
Copy link
Collaborator Author

Also, the error when exceeding deadline for flow control tokens is unhelpful. It should include all the information in the preceding log statement

Copy link

blathers-crl bot commented Nov 13, 2023

Hi @sumeerbhola, please add branch-* labels to identify which branch(es) this release-blocker affects.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@sumeerbhola sumeerbhola added branch-release-23.2 Used to mark GA and release blockers, technical advisories, and bugs for 23.2 and removed branch-release-23.2 Used to mark GA and release blockers, technical advisories, and bugs for 23.2 labels Nov 13, 2023
aadityasondhi added a commit to aadityasondhi/cockroach that referenced this issue Nov 29, 2023
This patch plumbs down a queue type identifier (`QueueKind`) into the
`WorkQueue` to help track down the origin of items in the queue.

Informs: cockroachdb#113990

Release note: None
aadityasondhi added a commit to aadityasondhi/cockroach that referenced this issue Nov 29, 2023
This patch now includes work priority in the `WorkQueue` traces and
error messages to better help track where work items originate from.

Informs cockroachdb#113990.

Release note: None
aadityasondhi added a commit to aadityasondhi/cockroach that referenced this issue Nov 29, 2023
This patch improves the error message when context deadline is reached in
flowcontrol wait queue. It includes priority, wait duration, stream.

Informs: cockroachdb#113990

Release note: None
craig bot pushed a commit that referenced this issue Nov 30, 2023
115213: clusterversion: add runbooks r=RaduBerinde a=RaduBerinde

This commit adds a `README.md` file which contains runbooks for
advancing versions.

Informs: #112629
Release note: None

Rendered file can be viewed here: https://github.com/RaduBerinde/cockroach/blob/version-doc/pkg/clusterversion/README.md

115262: admission: add extra info when deadline reached in flowcontrol r=sumeerbhola a=aadityasondhi

This patch improves the error message when context deadline is reached in flowcontrol wait queue. It includes priority, wait duration, stream.

Informs: #113990

Release note: None

115347: lease: skip TestRangefeedUpdatesHandledProperlyInTheFaceOfRaces under race r=rafiss a=rafiss

fixes #114324
Release note: None

Co-authored-by: Radu Berinde <[email protected]>
Co-authored-by: Aaditya Sondhi <[email protected]>
Co-authored-by: Rafi Shamim <[email protected]>
blathers-crl bot pushed a commit that referenced this issue Nov 30, 2023
This patch improves the error message when context deadline is reached in
flowcontrol wait queue. It includes priority, wait duration, stream.

Informs: #113990

Release note: None
@aadityasondhi aadityasondhi added branch-master Failures and bugs on the master branch. branch-release-23.2 Used to mark GA and release blockers, technical advisories, and bugs for 23.2 labels Dec 5, 2023
aadityasondhi added a commit to aadityasondhi/cockroach that referenced this issue Dec 5, 2023
This patch plumbs down a queue type identifier (`QueueKind`) into the
`WorkQueue` to help track down the origin of items in the queue.

Informs: cockroachdb#113990

Release note: None
aadityasondhi added a commit to aadityasondhi/cockroach that referenced this issue Dec 5, 2023
This patch now includes work priority in the `WorkQueue` traces and
error messages to better help track where work items originate from.

Informs cockroachdb#113990.

Release note: None
aadityasondhi added a commit to aadityasondhi/cockroach that referenced this issue Dec 6, 2023
This patch plumbs down a queue type identifier (`QueueKind`) into the
`WorkQueue` to help track down the origin of items in the queue.

Informs: cockroachdb#113990

Release note: None
aadityasondhi added a commit to aadityasondhi/cockroach that referenced this issue Dec 6, 2023
This patch now includes work priority in the `WorkQueue` traces and
error messages to better help track where work items originate from.

Informs cockroachdb#113990.

Release note: None
craig bot pushed a commit that referenced this issue Dec 7, 2023
114664: admission: include more work queue info in tracing and errors r=sumeerbhola a=aadityasondhi

See individual commits.

---
admission: inject queue kind into work queue tracing

This patch plumbs down a queue type identifier (`QueueKind`) into the
`WorkQueue` to help track down the origin of items in the queue.

Informs: #113990

Release note: None

---
admission: include work priority in trace and errors

This patch now includes work priority in the `WorkQueue` traces and
error messages to better help track where work items originate from.

Informs #113990.

Release note: None


115737: sql: add ULID entropy to eval context r=rafiss a=rafiss

Rather than making a new entropy source for each invocation, now it is
shared for all usages of the same eval context.

```
goos: darwin
goarch: arm64
                                        │    old.txt    │               new.txt                │
                                        │    sec/op     │    sec/op     vs base                │
GenerateID/gen_random_uuid/rows=1-10        82.94µ ± 1%   82.97µ ±  1%        ~ (p=0.870 n=10)
GenerateID/gen_random_uuid/rows=10-10       91.06µ ± 1%   91.22µ ±  1%        ~ (p=0.631 n=10)
GenerateID/gen_random_uuid/rows=100-10      164.8µ ± 3%   164.9µ ±  1%        ~ (p=1.000 n=10)
GenerateID/gen_random_uuid/rows=1000-10     926.2µ ± 3%   926.7µ ±  5%        ~ (p=0.529 n=10)
GenerateID/gen_random_ulid/rows=1-10       101.88µ ± 1%   80.07µ ±  1%  -21.41% (p=0.000 n=10)
GenerateID/gen_random_ulid/rows=10-10      223.72µ ± 9%   85.15µ ± 33%  -61.94% (p=0.000 n=10)
GenerateID/gen_random_ulid/rows=100-10     1174.6µ ± 1%   120.5µ ± 39%  -89.74% (p=0.000 n=10)
GenerateID/gen_random_ulid/rows=1000-10   10386.2µ ± 1%   464.5µ ±  8%  -95.53% (p=0.000 n=10)
geomean                                     365.8µ        160.5µ        -56.11%

                                        │    old.txt     │                new.txt                │
                                        │      B/op      │     B/op       vs base                │
GenerateID/gen_random_uuid/rows=1-10       32.27Ki ±  6%   32.28Ki ±  7%        ~ (p=1.000 n=10)
GenerateID/gen_random_uuid/rows=10-10      34.57Ki ±  6%   34.55Ki ±  6%        ~ (p=0.529 n=10)
GenerateID/gen_random_uuid/rows=100-10     49.94Ki ±  0%   49.91Ki ±  0%        ~ (p=0.382 n=10)
GenerateID/gen_random_uuid/rows=1000-10    318.9Ki ± 46%   319.3Ki ± 61%        ~ (p=0.796 n=10)
GenerateID/gen_random_ulid/rows=1-10       40.52Ki ±  0%   32.16Ki ±  0%  -20.63% (p=0.000 n=10)
GenerateID/gen_random_ulid/rows=10-10      80.37Ki ±  7%   34.46Ki ±  5%  -57.12% (p=0.000 n=10)
GenerateID/gen_random_ulid/rows=100-10    470.89Ki ±  0%   49.81Ki ±  0%  -89.42% (p=0.000 n=10)
GenerateID/gen_random_ulid/rows=1000-10   4489.5Ki ±  0%   318.4Ki ±  4%  -92.91% (p=0.000 n=10)
geomean                                    136.7Ki         64.84Ki        -52.59%

                                        │    old.txt    │               new.txt                │
                                        │   allocs/op   │  allocs/op    vs base                │
GenerateID/gen_random_uuid/rows=1-10        405.0 ±  9%    405.0 ±  9%        ~ (p=0.940 n=10)
GenerateID/gen_random_uuid/rows=10-10       480.0 ±  1%    480.0 ±  0%        ~ (p=0.463 n=10)
GenerateID/gen_random_uuid/rows=100-10      975.0 ±  0%    975.0 ±  0%        ~ (p=0.685 n=10)
GenerateID/gen_random_uuid/rows=1000-10    7.041k ± 18%   7.041k ± 17%        ~ (p=0.639 n=10)
GenerateID/gen_random_ulid/rows=1-10        411.0 ±  0%    405.0 ±  0%   -1.46% (p=0.000 n=10)
GenerateID/gen_random_ulid/rows=10-10       513.5 ± 18%    480.0 ±  8%   -6.52% (p=0.006 n=10)
GenerateID/gen_random_ulid/rows=100-10     1281.5 ±  0%    975.0 ±  0%  -23.92% (p=0.000 n=10)
GenerateID/gen_random_ulid/rows=1000-10   10.062k ±  0%   7.040k ±  3%  -30.03% (p=0.000 n=10)
geomean                                    1.175k         1.075k         -8.52%
```

fixes #115666
Release note (performance improvement): Improved the performance of the gen_random_ulid builtin function.

Co-authored-by: Aaditya Sondhi <[email protected]>
Co-authored-by: Rafi Shamim <[email protected]>
aadityasondhi added a commit to aadityasondhi/cockroach that referenced this issue Dec 7, 2023
This patch plumbs down a queue type identifier (`QueueKind`) into the
`WorkQueue` to help track down the origin of items in the queue.

Informs: cockroachdb#113990

Release note: None
aadityasondhi added a commit to aadityasondhi/cockroach that referenced this issue Dec 7, 2023
This patch now includes work priority in the `WorkQueue` traces and
error messages to better help track where work items originate from.

Informs cockroachdb#113990.

Release note: None
aadityasondhi added a commit to aadityasondhi/cockroach that referenced this issue Dec 7, 2023
This patch plumbs down a queue type identifier (`QueueKind`) into the
`WorkQueue` to help track down the origin of items in the queue.

Informs: cockroachdb#113990

Release note: None
aadityasondhi added a commit to aadityasondhi/cockroach that referenced this issue Dec 7, 2023
This patch now includes work priority in the `WorkQueue` traces and
error messages to better help track where work items originate from.

Informs cockroachdb#113990.

Release note: None
aadityasondhi added a commit to aadityasondhi/cockroach that referenced this issue Dec 10, 2023
This patch plumbs down a queue type identifier (`QueueKind`) into the
`WorkQueue` to help track down the origin of items in the queue.

Informs: cockroachdb#113990

Release note: None
aadityasondhi added a commit to aadityasondhi/cockroach that referenced this issue Dec 10, 2023
This patch now includes work priority in the `WorkQueue` traces and
error messages to better help track where work items originate from.

Informs cockroachdb#113990.

Release note: None
aadityasondhi added a commit to aadityasondhi/cockroach that referenced this issue Dec 10, 2023
This patch plumbs down a queue type identifier (`QueueKind`) into the
`WorkQueue` to help track down the origin of items in the queue.

Informs: cockroachdb#113990

Release note: None
aadityasondhi added a commit to aadityasondhi/cockroach that referenced this issue Dec 10, 2023
This patch now includes work priority in the `WorkQueue` traces and
error messages to better help track where work items originate from.

Informs cockroachdb#113990.

Release note: None
@aadityasondhi
Copy link
Collaborator

Since both #114664 and #115262 have been merged and backported to relevant branches, this issue is now resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-admission-control branch-master Failures and bugs on the master branch. branch-release-23.2 Used to mark GA and release blockers, technical advisories, and bugs for 23.2 C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) GA-blocker T-admission-control Admission Control
Projects
None yet
Development

No branches or pull requests

2 participants