Wr/pdr feat requests #675

WillieRuemmele · 2023-06-27T21:35:10Z

What does this PR do?

adds testing abilities to deploy report/validate
adds destructive changes to deploy validate
~~makes --use-most-recent the default behavior for report/resume/cancel/quick~~
deploy report --results-dir now overrides the dir passed when initializing the deploy, previously it wouldn't do anything

What issues does this PR fix or reference?

@W-13665551@
forcedotcom/cli#2265

@W-13640624@
forcedotcom/cli#2246

@W-13658630@
forcedotcom/cli#2255

…s to validate

cristiand391 · 2023-07-04T12:51:56Z

test/commands/deploy/metadata/cancel.nut.ts

@@ -64,6 +64,31 @@ describe('deploy metadata cancel NUTs', () => {
        expect(cancel.jsonOutput.name).to.equal('CannotCancelDeployError');
      }
    });
+
+    it.skip('should cancel most recently started deployment without specifying the flag', () => {


there are a few of the new NUTs in this PR being skipped.

cristiand391

LGTM.
Just need to ~~remove it.skip from new NUTs~~ (nvm, just saw the commit title where it was added 😄) fix merge conflicts.

cristiand391 · 2023-07-04T15:16:36Z

QA notes:

adds testing abilities to `deploy report/validate`

setup:
create a scratch org, try to deploy dreamhouse project

✅ set results-dir and coverage format at async deploy, then change them in deploy report

➜  dreamhouse-lwc git:(main) sf project deploy start --source-dir force-app --test-level RunAllTestsInOrg --results-dir gh-test-async --async --coverage-formatters html
Deploying v58.0 metadata to [email protected] using the v58.0 SOAP API.
Deploy ID: 0Af8K00000Nk9TjSAJ
Warning: You requested an async deploy with code coverage or JUnit results. The reports will be available when the deploy completes.
Deploy has been queued.
                                                                                                                                                                          
Run "sf project deploy resume --job-id 0Af8K00000Nk9TjSAJ" to resume watching the deploy.
Run "sf project deploy report --job-id 0Af8K00000Nk9TjSAJ" to get the latest status.
Run "sf project deploy cancel --job-id 0Af8K00000Nk9TjSAJ" to cancel the deploy.

➜  dreamhouse-lwc git:(main) ✗ ../sf/plugin-deploy-retrieve/bin/dev project:deploy:report --job-id 0Af8K00000Nk9TjSAJ --results-dir gh-test-async-lala --coverage-formatte
rs json --junit
0Af8K00000Nk9TjSAJ... Succeeded

Deploy Info
===================================================
| Key                      Value
| ──────────────────────── ────────────────────────
| checkOnly                false
| completedDate            2023-07-04T14:35:09.000Z
| createdBy                0058K000002XTaz
| createdByName            User User
| createdDate              2023-07-04T14:30:58.000Z
| done                     true
| id                       0Af8K00000Nk9TjSAJ
| ignoreWarnings           false
| lastModifiedDate         2023-07-04T14:35:09.000Z
| numberComponentErrors    0
| numberComponentsDeployed 91
| numberComponentsTotal    91
| numberTestErrors         0
| numberTestsCompleted     11
| numberTestsTotal         11
| rollbackOnError          true
| runTestsEnabled          true
| startDate                2023-07-04T14:34:29.000Z
| status                   Succeeded
| success                  true

Deploy Options
================================================================================
| Key                 Value
| ─────────────────── ──────────────────────────────────────────────────────────
| test-level          RunAllTestsInOrg
| results-dir         gh-test-async-lala
| async               true
| coverage-formatters [ 'json' ]
| dry-run             false
| ignore-conflicts    false
| ignore-errors       false
| ignore-warnings     false
| target-org          [email protected]
| api                 SOAP
| manifest            /Users/cdominguez/.sf/manifestCache/0Af8K00000Nk9TjSAJ.xml
| wait                33
| isMdapi             false
| job-id              0Af8K00000Nk9TjSAJ
| junit               true

Deployed Source
===============================================================================================================================================================================
| State   Name                                Type                     Path
| ─────── ─────────────────────────────────── ──────────────────────── ────────────────────────────────────────────────────────────────────────────────────────────────────────
| Created FileUtilities                       ApexClass                force-app/main/default/classes/FileUtilities.cls
...
Test Results Summary
Passing: 11
Failing: 0
Total: 11
Time: 4030
Code Coverage formats, [json], written to gh-test-async-lala/
Junit results written to gh-test-async-lala/junit/junit.xml

➜  dreamhouse-lwc git:(main) ✗ tree gh-test-async-lala/
gh-test-async-lala/
├── coverage
│   └── coverage.json
└── junit
    └── junit.xml

3 directories, 2 files

👎🏼 project deploy report --results-dir <dir> --coverage-formatters html requires to specify --junit flag.
deploy start doesn't have this requirement.
suggestion: make --results-dir flag to require either --coverage-formatters or --junit or both.
Both --results-dir lala --junit and --results-dir lala --coverage-formatters html cases should work without having to specify each other.

➜  dreamhouse-lwc git:(main) ✗ ../sf/plugin-deploy-retrieve/bin/dev project:deploy:report \
--job-id 0Af8K00000Nk9TjSAJ \
--results-dir gh-test-async-lala \
--coverage-formatters json
Error (1): The following error occurred:
  All of the following must be provided when using --results-dir: --junit, --coverage-formatters
See more help with --help

✅ project deploy validate outputs code coverage

➜  dreamhouse-lwc git:(main) sf project deploy validate --test-level RunAllTestsInOrg --coverage-formatters html --source-dir force-app --results-dir validate-cov --conci
se
Validating Deployment of v58.0 metadata to [email protected] using the v58.0 SOAP API.
Deploy ID: 0AfDa00001SPNfZKAX
Status: Succeeded | ████████████████████████████████████████ | 91/91 Components (Errors:0) | 11/11 Tests (Errors:0)

Test Results Summary
Passing: 11
Failing: 0
Total: 11
Time: 2599
Code Coverage formats, [html], written to validate-cov/

Successfully validated the deployment (0AfDa00001SPNfZKAX).
Run "sf project deploy quick --job-id 0AfDa00001SPNfZKAX" to execute this deploy
➜  dreamhouse-lwc git:(main) t validate-cov
validate-cov
└── coverage
    └── html
        ├── FileUtilities.html
        ├── GeocodingService.html
        ├── PagedResult.html
        ├── PropertyController.html
        ├── SampleDataController.html
        ├── base.css
        ├── block-navigation.js
        ├── favicon.png
        ├── index.html
        ├── prettify.css
        ├── prettify.js
        ├── sort-arrow-sprite.png
        └── sorter.js

3 directories, 13 files

adds destructive changes to deploy validate

✅

➜  dreamhouse-lwc git:(main) ✗ sf project deploy validate --manifest package.xml --post-destructive-changes destructiveChangesPost.xml
Validating Deployment of v58.0 metadata to [email protected] using the v58.0 SOAP API.
Deploy ID: 0AfDH00001i1hoF0AQ
Status: Succeeded | ████████████████████████████████████████ | 2/2 Components (Errors:0) | 11/11 Tests (Errors:0)

Validated Source
===================================================================================================
| State     Name             Type      Path
| ───────── ──────────────── ───────── ────────────────────────────────────────────────────────────
| Unchanged GeocodingService ApexClass force-app/main/default/classes/GeocodingService.cls
| Unchanged GeocodingService ApexClass force-app/main/default/classes/GeocodingService.cls-meta.xml
| Deleted   myApexClass      ApexClass force-app/main/default/classes/myApexClass.cls
| Deleted   myApexClass      ApexClass force-app/main/default/classes/myApexClass.cls-meta.xml

Deleted Source
===============================================================================
| Name        Type      Path
| ─────────── ───────── ───────────────────────────────────────────────────────
| myApexClass ApexClass force-app/main/default/classes/myApexClass.cls
| myApexClass ApexClass force-app/main/default/classes/myApexClass.cls-meta.xml

Test Results Summary
Passing: 11
Failing: 0
Total: 11
Time: 3214

Successfully validated the deployment (0AfDH00001i1hoF0AQ).
Run "sf project deploy quick --job-id 0AfDH00001i1hoF0AQ" to execute this deploy
➜  dreamhouse-lwc git:(main) ✗ sf project deploy quick --job-id 0AfDH00001i1hoF0AQ
Deploy ID: 0AfDH00001i1hoF0AQ

Validated Source
===================================================================
| State     Name             Type      Path
| ───────── ──────────────── ───────── ────────────────────────────
| Unchanged GeocodingService ApexClass classes/GeocodingService.cls
| Deleted   myApexClass      ApexClass classes/myApexClass.cls

Deleted Source
===============================================
| Name        Type      Path
| ─────────── ───────── ───────────────────────
| myApexClass ApexClass classes/myApexClass.cls

Test Results Summary
Passing: 11
Failing: 0
Total: 11
Time: 3214

Successfully deployed (0AfDH00001i1hoF0AQ).
➜  dreamhouse-lwc git:(main) ✗ sf data query -q "SELECT IsNameObsolete FROM SourceMember WHERE MemberType='ApexClass' AND MemberName='myApexClass'" --use-tooling-api
 ISNAMEOBSOLETE
 ──────────────
 true
Total number of records retrieved: 1.
Querying Data... done

src/commands/project/deploy/report.ts

cristiand391 · 2023-07-05T18:23:51Z

QA notes:

👎🏼 project deploy report --results-dir
--coverage-formatters html requires to specify --junit flag.
deploy start doesn't have this requirement.
suggestion: make --results-dir flag to require either --coverage-formatters or --junit or both.
Both --results-dir lala --junit and --results-dir lala --coverage-formatters html cases should work without having to specify each other.

✅ --results-dir <dir> without --junit or --coverage fails

➜  dreamhouse-lwc git:(main) ✗ ../sf/plugin-deploy-retrieve/bin/dev project:deploy:validate --test-level RunAllTestsInOrg \
--source-dir force-app \
--results-dir validate-cov \
--concise
Error (1): The following error occurred:
  One of the following must be provided when using --results-dir: --coverage-formatters, --junit
See more help with --help

✅ --junit works without --coverage-formatters and respects --results-dir

➜  dreamhouse-lwc git:(main) ✗ ../sf/plugin-deploy-retrieve/bin/dev project:deploy:validate --test-level RunAllTestsInOrg \
--source-dir force-app \
--junit \
--results-dir validate-cov \
--concise
Validating Deployment of v58.0 metadata to [email protected] using the v58.0 SOAP API.
Deploy ID: 0AfDa00001SaeB9KAJ
Status: Succeeded | ████████████████████████████████████████ | 91/91 Components (Errors:0) | 11/11 Tests (Errors:0)

Test Results Summary
Passing: 11
Failing: 0
Total: 11
Time: 2623
Junit results written to validate-cov/junit/junit.xml

Successfully validated the deployment (0AfDa00001SaeB9KAJ).
Run "sf project deploy quick --job-id 0AfDa00001SaeB9KAJ" to execute this deploy
➜  dreamhouse-lwc git:(main) ✗ ll validate-cov/junit
total 8
-rw-r--r--@ 1 cdominguez  staff   2.4K Jul  5 15:18 junit.xml

✅ --coverage-formatter work without --junit and respects --results-dir

➜  dreamhouse-lwc git:(main) ../sf/plugin-deploy-retrieve/bin/dev project:deploy:validate --test-level RunAllTestsInOrg \
--source-dir force-app \
--coverage-formatters json \
--results-dir validate-cov \
--concise
Validating Deployment of v58.0 metadata to [email protected] using the v58.0 SOAP API.
Deploy ID: 0AfDa00001SaeDKKAZ
Status: Succeeded | ████████████████████████████████████████ | 91/91 Components (Errors:0) | 11/11 Tests (Errors:0)

Test Results Summary
Passing: 11
Failing: 0
Total: 11
Time: 2722
Code Coverage formats, [json], written to validate-cov/

Successfully validated the deployment (0AfDa00001SaeDKKAZ).
Run "sf project deploy quick --job-id 0AfDa00001SaeDKKAZ" to execute this deploy
➜  dreamhouse-lwc git:(main) t validate-cov
validate-cov
└── coverage
    └── coverage.json

2 directories, 1 file

✅ --junit and --coverage-formatters work without --results-dir, coverage folder defaults to coverage (default value handled in formatter)

➜  dreamhouse-lwc git:(main) ../sf/plugin-deploy-retrieve/bin/dev project:deploy:validate --test-level RunAllTestsInOrg \
--source-dir force-app \
--coverage-formatters json \
--junit \
--concise
Validating Deployment of v58.0 metadata to [email protected] using the v58.0 SOAP API.
Deploy ID: 0AfDa00001SaeFQKAZ
Status: Succeeded | ████████████████████████████████████████ | 91/91 Components (Errors:0) | 11/11 Tests (Errors:0)

Test Results Summary
Passing: 11
Failing: 0
Total: 11
Time: 3000
Code Coverage formats, [json], written to coverage/
Junit results written to coverage/junit/junit.xml

Successfully validated the deployment (0AfDa00001SaeFQKAZ).
Run "sf project deploy quick --job-id 0AfDa00001SaeFQKAZ" to execute this deploy
➜  dreamhouse-lwc git:(main) t coverage
coverage
├── coverage
│   └── coverage.json
└── junit
    └── junit.xml

3 directories, 2 files

WillieRuemmele added 2 commits June 27, 2023 14:43

fix: add standard test flags to validate/report, add destructive flag…

39b3d76

…s to validate

chore: revert --use-most-recent default change, skip NUTs

7279ceb

WillieRuemmele force-pushed the wr/pdrFeatRequests branch from d40ced5 to 7279ceb Compare June 28, 2023 20:49

test: fix NUT

601fc2b

cristiand391 reviewed Jul 4, 2023

View reviewed changes

cristiand391 requested changes Jul 4, 2023

View reviewed changes

WillieRuemmele added 2 commits July 5, 2023 09:35

chore: fix flag validation for report

723e8b8

chore: resolve conflicts

ad434aa

WillieRuemmele marked this pull request as ready for review July 5, 2023 15:37

cristiand391 reviewed Jul 5, 2023

View reviewed changes

src/commands/project/deploy/report.ts Outdated Show resolved Hide resolved

chore: results-dir relationsip logic

0d51437

cristiand391 approved these changes Jul 5, 2023

View reviewed changes

cristiand391 merged commit d16bf99 into main Jul 5, 2023

cristiand391 deleted the wr/pdrFeatRequests branch July 5, 2023 18:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Wr/pdr feat requests #675

Wr/pdr feat requests #675

WillieRuemmele commented Jun 27, 2023 •

edited

Loading

cristiand391 Jul 4, 2023

cristiand391 left a comment •

edited

Loading

cristiand391 commented Jul 4, 2023 •

edited

Loading

cristiand391 commented Jul 5, 2023

Wr/pdr feat requests #675

Wr/pdr feat requests #675

Conversation

WillieRuemmele commented Jun 27, 2023 • edited Loading

What does this PR do?

What issues does this PR fix or reference?

cristiand391 Jul 4, 2023

Choose a reason for hiding this comment

cristiand391 left a comment • edited Loading

Choose a reason for hiding this comment

cristiand391 commented Jul 4, 2023 • edited Loading

adds testing abilities to deploy report/validate

adds destructive changes to deploy validate

cristiand391 commented Jul 5, 2023

WillieRuemmele commented Jun 27, 2023 •

edited

Loading

cristiand391 left a comment •

edited

Loading

cristiand391 commented Jul 4, 2023 •

edited

Loading

adds testing abilities to `deploy report/validate`