feat: add code coverage and junit reporting #491

peternhale · 2022-05-20T13:45:22Z

What does this PR do?

Add code coverage and junit reporting for mdapi deploy, deploy report, source deploy and source deploy report commands

What issues does this PR fix or reference?

@AW-11025210@

@

src/deployCommand.ts

src/commands/force/source/deploy.ts

messages/deploy.json

messages/md.deploy.json

messages/md.deployreport.json

messages/report.json

messages/deploy.json

messages/md.deploy.json

messages/md.deployreport.json

messages/report.json

Co-authored-by: Juliet Shackell <[email protected]>

src/coverageUtils.ts

mshanemc · 2022-05-20T20:17:48Z

src/deployCommand.ts

 import { SourceCommand } from './sourceCommand';
 import { DeployData, Stash } from './stash';
+import { transformCoverageToApexCoverage, transformDeployTestsResultsToTestResult } from './coverageUtils';
+// TODO: this function needs to be moved to a shared location
+import { toArray } from './formatters/resultFormatter';


mabe use this https://github.com/forcedotcom/source-deploy-retrieve/blob/fa6006b54434592cc1c143167addd5dde01fb45e/src/utils/collections.ts#L14

I chose not to switch. SDR does not formally export that function. To me that means it is not meant to be consumed by others.

mshanemc · 2022-05-20T20:19:16Z

src/deployCommand.ts

@@ -162,6 +181,85 @@ export abstract class DeployCommand extends SourceCommand {

    return this.isAsync ? this.report(validatedDeployId) : this.poll(validatedDeployId);
  }
+
+  protected createRequestedReports(): void {


I like to name these kind of things maybeCreateRequestedReports so that when you're reading it from the command, it's obvious that it might no-op.

mshanemc · 2022-05-20T20:25:37Z

src/deployCommand.ts

+    coverageReport.generateReports();
+  }
+
+  protected getCoverageFormattersOptions(formatters: string[] = []): CoverageReporterOptions {


if there are only certain strings that would be valid, you could use the typings here. Then, increateRequestedReports, you can use this.getFlag<CoverageReportFormats[]>(this.flags.coverageformatters)

Suggested change

protected getCoverageFormattersOptions(formatters: string[] = []): CoverageReporterOptions {

protected getCoverageFormattersOptions(formatters: CoverageReportFormats[] = []): CoverageReporterOptions {

src/deployCommand.ts

mshanemc · 2022-05-23T12:31:48Z

src/deployCommand.ts

+    const junitResults = jUnitReporter.format(testResult);
+
+    const junitReportPath = path.join(this.outputDir, 'junit');
+    fs.mkdirSync(junitReportPath, { recursive: true });


for this, and createCoverageReport I'd prefer that methods with fs ops be async and use the async fs.

I'll look into switch to async

To make this async would require a larger refactor. If that is to be done, I suggest a separate work item or wait until these commands appear in sf.

Co-authored-by: Shane McLaughlin <[email protected]>

messages/md.deployreport.json

messages/report.json

Co-authored-by: Benjamin <[email protected]>

Co-authored-by: Juliet Shackell <[email protected]>

maggiben · 2022-05-25T16:09:31Z

Coverage report for dreamhouse-lwc:

=== Test Results Summary
Passing: 7
Failing: 0
Total: 7
Time: 0

Sample classes:

GeocodingService w/100% coverage:

<file name="GeocodingService" path="no-map/GeocodingService">
      <metrics statements="1" coveredstatements="1" conditionals="0" coveredconditionals="0" methods="0" coveredmethods="0"/>
      <line num="Infinity" count="1" type="stmt"/>
    </file>

PropertyController w/95% coverage:

<file name="PropertyController" path="no-map/PropertyController">
      <metrics statements="46" coveredstatements="44" conditionals="0" coveredconditionals="0" methods="0" coveredmethods="0"/>
      <line num="26" count="0" type="stmt"/>
      <line num="27" count="1" type="stmt"/>
      <line num="28" count="1" type="stmt"/>
      <line num="29" count="1" type="stmt"/>
      <line num="30" count="1" type="stmt"/>
      <line num="31" count="0" type="stmt"/>
      <line num="32" count="1" type="stmt"/>
      <line num="33" count="1" type="stmt"/>
      <line num="34" count="1" type="stmt"/>
      <line num="35" count="1" type="stmt"/>
      <line num="36" count="1" type="stmt"/>
      <line num="37" count="1" type="stmt"/>
      <line num="38" count="1" type="stmt"/>
      <line num="39" count="1" type="stmt"/>
      <line num="40" count="1" type="stmt"/>
      <line num="41" count="1" type="stmt"/>
      <line num="42" count="1" type="stmt"/>
      <line num="43" count="1" type="stmt"/>
      <line num="44" count="1" type="stmt"/>
      <line num="45" count="1" type="stmt"/>
      <line num="46" count="1" type="stmt"/>
      <line num="47" count="1" type="stmt"/>
      <line num="48" count="1" type="stmt"/>
      <line num="49" count="1" type="stmt"/>
      <line num="50" count="1" type="stmt"/>
      <line num="51" count="1" type="stmt"/>
      <line num="52" count="1" type="stmt"/>
      <line num="53" count="1" type="stmt"/>
      <line num="54" count="1" type="stmt"/>
      <line num="55" count="1" type="stmt"/>
      <line num="56" count="1" type="stmt"/>
      <line num="57" count="1" type="stmt"/>
      <line num="58" count="1" type="stmt"/>
      <line num="59" count="1" type="stmt"/>
      <line num="60" count="1" type="stmt"/>
      <line num="61" count="1" type="stmt"/>
      <line num="62" count="1" type="stmt"/>
      <line num="63" count="1" type="stmt"/>
      <line num="64" count="1" type="stmt"/>
      <line num="65" count="1" type="stmt"/>
      <line num="66" count="1" type="stmt"/>
      <line num="67" count="1" type="stmt"/>
      <line num="68" count="1" type="stmt"/>
      <line num="69" count="1" type="stmt"/>
      <line num="70" count="1" type="stmt"/>
      <line num="71" count="1" type="stmt"/>
    </file>

*(Note line 26 & 31 ternary is not covered)

maggiben · 2022-05-25T17:19:01Z

Class with 0% coverage:

    <file name="myApexClass" path="no-map/myApexClass">
      <metrics statements="16" coveredstatements="8" conditionals="0" coveredconditionals="0" methods="0" coveredmethods="0"/>
      <line num="3" count="0" type="stmt"/>
      <line num="4" count="1" type="stmt"/>
      <line num="5" count="1" type="stmt"/>
      <line num="6" count="0" type="stmt"/>
      <line num="7" count="0" type="stmt"/>
      <line num="8" count="0" type="stmt"/>
      <line num="9" count="0" type="stmt"/>
      <line num="10" count="0" type="stmt"/>
      <line num="11" count="0" type="stmt"/>
      <line num="12" count="1" type="stmt"/>
      <line num="13" count="0" type="stmt"/>
      <line num="14" count="1" type="stmt"/>
      <line num="15" count="1" type="stmt"/>
      <line num="16" count="1" type="stmt"/>
      <line num="17" count="1" type="stmt"/>
      <line num="18" count="1" type="stmt"/>
    </file>

messages/deploy.json

messages/deployCommand.json

messages/md.deploy.json

jshackell-sfdc

See my suggestions.

Co-authored-by: Juliet Shackell <[email protected]>

maggiben · 2022-05-26T17:16:34Z

Testing Levels for `source:deploy`:

✅ force:source:deploy with testlevel = NoTestRun produces no results
✅ force:source:deploy with testlevel = RunSpecifiedTests produces results associated with theruntests passed to the command and writes coverage and junit results to resultsdir
✅ force:source:deploy with testlevel = RunLocalTests writes coverage and junit results to resultsdir
✅ force:source:deploy with testlevel = RunAllTestsInOrg produces test coverage and junit summary and results are written to the resultsdir

Asynchronous warning message for `source:deploy` & `mdapi:deploy`:

✅ Warning: You requested an async deploy with coverage or junit. The reports will not be available until the deploy completes. is displayed

Testing Levels for `mdapi:deploy`:

✅ force:mdapi:deploy with testlevel = NoTestRun produces no results
❌ force:source:deploy with testlevel = RunSpecifiedTests fails to deploy (fails to meet the coverage expectations in the org) but this is unrelated to this change.
✅ force:mdapi:deploy with testlevel = RunLocalTests writes coverage and junit results to resultsdir
✅ force:mdapi:deploy with testlevel = RunAllTestsInOrg writes coverage and junit results to resultsdir

peternhale added 2 commits May 19, 2022 14:55

feat: add code coverage and junt reports to deploy and deploy report

2e26ef9

@

chore: ensure miss when looking for source files

62349f8

peternhale requested review from mshanemc and mdonnalley May 20, 2022 13:45

mdonnalley reviewed May 20, 2022

View reviewed changes

src/deployCommand.ts Outdated Show resolved Hide resolved

src/deployCommand.ts Outdated Show resolved Hide resolved

src/deployCommand.ts Outdated Show resolved Hide resolved

maggiben reviewed May 20, 2022

View reviewed changes

src/commands/force/source/deploy.ts Outdated Show resolved Hide resolved

peternhale added 2 commits May 20, 2022 12:01

chore: apply review comments

6f21057

chore: apply review changes requests

6314a2e

peternhale requested review from maggiben and mdonnalley May 20, 2022 18:13

mdonnalley approved these changes May 20, 2022

View reviewed changes

maggiben approved these changes May 20, 2022

View reviewed changes