test: collect detailed logs for tests in datadog[infeng-752] #9637
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for determined-ui ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #9637 +/- ##
=======================================
Coverage 53.32% 53.32%
=======================================
Files 1254 1254
Lines 152655 152655
Branches 3244 3244
=======================================
+ Hits 81398 81399 +1
+ Misses 71105 71104 -1
Partials 152 152
Flags with carried forward coverage won't be shown. Click here to find out more. |
djanicekpach
force-pushed
the
djanicek/infeng-752/local-agent
branch
from
5336074
to
a968a33
Compare
loksonarius
approved these changes
Jul 16, 2024
loksonarius
reviewed
Jul 16, 2024
loksonarius
approved these changes
Jul 16, 2024
This reverts commit e638edd.
djanicekpach
force-pushed
the
djanicek/infeng-752/local-agent
branch
from
8091c01
to
46855eb
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
test: collect detailed logs for tests in datadog[infeng-752]
Ticket
INFENG-752
Description
This PR adds a datadog agent to E2E tests so that we can collect logs from devcluster and any dependency containers running on the machine. Each of these does count towards our billed host count in DataDog so I'll be pretty cautious around this.
The
AIS_DD_ENABLE_MONITORINGflag can turn this off for all pipelines if we need it. I'll be monitoring pipeline closely on merge-day. I used an ENV variable rather than a pipeline parameter since it is harder to turn things off quickly with pipeline parameters, and they would have required threading a new parameter through half a dozen jobs.I also created some dashboards in datadog to help with the ease-of-use. See the testing section.
Test Plan
Ran CI and confirmed results showed in datadog when appropriate. Here's what you can see:
Go to the branch test dashboard for my branch: https://app.datadoghq.com/dashboard/vra-jm2-diq/branch-test-info?fromUser=true&refresh_mode=sliding&tpl_var_branch%5B0%5D=djanicek%2Finfeng-752%2Flocal-agent&view=spans&from_ts=1720107273963&to_ts=1720712073963&live=true
You can
View Test/Host Detailsby clicking on a test in the first timeseries graph:This will load a test dashboard: https://app.datadoghq.com/dashboard/spd-5rs-z7w?=&fromUser=false&refresh_mode=sliding&tpl_var_TestName%5B0%5D=%40test.name%3Atest_post_user_api&view=spans&from_ts=1720107353085&to_ts=1720712153085&live=true
You can use the
Test Host Informationto get host info.You may need to tweak the lookback time. If it's been more than 3 days logs will be gone due to our retention settings, but if not, devcluster, postgres, and other container logs should be available:
Checklist
docs/release-notes/See Release Note for details.