JUnit output missing filename

[ASHR4]

Using Azure DevOps to publish test results in the format of JUNIT using this command to generate output:

$Output = docker run --rm -v "$($INPUT_WORK_DIR):/src" `
                    --entrypoint=/bin/sh ghcr.io/terraform-linters/tflint-bundle `
                    -c "tflint -v; tflint --init --config '/src/$INPUT_TFLINT_CONFIG'; tflint --config '/src/$INPUT_TFLINT_CONFIG' '/src/$tf_directory' --format='junit'" | Out-String
            $Output | Out-File -FilePath "$($INPUT_WORK_DIR)/TFLint/TFLint_$guid.xml"

I then Publish this task using:

- task: PublishTestResults@2
            condition: always()
            inputs:
              testRunTitle: "TFLint Analysis Results"
              failTaskOnFailedTests: false
              publishRunAttachments: true
              mergeTestResults: true
              testResultsFormat: "JUnit"
              testResultsFiles: |
                **/*.xml
              searchFolder: "$(System.DefaultWorkingDirectory)/TFLint"
            displayName: "Publish TFLint Results"

All of the above works and publishes the tests with no issues.

The issue comes from the XML it generates where the classname is the path in which has been scanned - the issue with this is that the Publish Test Task uses this:

This causes the test title to be the error message and not the file path or a combination - I suppose I would like an amended
failure message to include the path it has scanned:

line 54, col 1, FIle [Path] Notice - `data_vnet` variable has no description (terraform_documented_variables)

I know I'm a small use case for this, I was hoping if anyone would have any advice or if this can be updated?

[bendrucker]

See #1409 which fixes the misses filename in the failure message (a bug)

[bendrucker]

For reference: https://eslint.org/docs/user-guide/formatters/#junit

ESLint considers each file its own <testsuite> with 1 test case per linter error, with name set to the rule ID. As far as the repetition in your screenshot, that would seem to be the case with other linters' JUnit output too. It's a bit awkward to put lint results into JUnit but the rule name in name and the full failure message in <failure> seems like the natural choice.

[ASHR4]

Hi @bendrucker,

I saw #1409 has been merged and I've just downloaded the latest TFlint version:

$ tflint -v
TFLint version 0.37.0

I can now see "type" but it's empty (can't remember if it was always like this) and no filename is within the failure message?

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite tests="1" failures="1" time="0" name="">
    <properties></properties>
    <testcase classname="\terraform\modules\tf_resource_group\resource\main.tf" name="terraform_required_providers" time="0">
      <failure message="Missing version constraint for provider &#34;azurerm&#34; in &#34;required_providers&#34;" type="">line 1, col 1, Warning - Missing version constraint for provider &#34;azurerm&#34; in &#34;required_providers&#34; (terraform_required_providers)</failure>
    </testcase>
  </testsuite>
</testsuites>

Any recommendations in which I havent taken into account? Unsure what your release process is I.e next version will include this :)

Thanks

[bendrucker]

Not released yet

[ASHR4]

Not released yet

You're a star for this - appreciate the help!

[ASHR4]

Hi @bendrucker,

I can see the new update and it's working as expected. I just had a question if an additional feature can be implemented on top of this.

It's great I know which file TFLint is talking about, but for my projects and possible other users, I scan every terraform directory within my codebase to be linted. Each of these directories pretty much have the same files I.e backend.tf + Main.tf etc

Would it be possible to get the literal path of the file so instead of "backend.tf" it becomes "/made/up/path/backend.tf" or even just it's parent dir so I know where i am I.e "/project1/backend.tf"

Let me know if this would be useful and if not I can find a solution that works for my specific use case 😁

[bendrucker]

No, absolute paths are not an option. Nor is the parent directory.

Please reserve issues for a specific proposal and avoid reopening them for use as an ongoing support thread.