Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to get number of errors with regex_number #3446

Closed
alexanderbazhenoff opened this issue Mar 23, 2024 · 16 comments
Closed

Unable to get number of errors with regex_number #3446

alexanderbazhenoff opened this issue Mar 23, 2024 · 16 comments
Labels
bug Something isn't working

Comments

@alexanderbazhenoff
Copy link
Contributor

alexanderbazhenoff commented Mar 23, 2024

Hi, not so serious bug, but...

Describe the bug
Here is a actions log where trufflehog pass, but still trying to capture number of errors.

 ##[group]�[32m✅ Linted [REPOSITORY] files with [trufflehog] successfully - (10.5s)�[0m (expand for details)
 - Using [trufflehog v3.71.0] https://megalinter.io/7.10.0/descriptors/repository_trufflehog
 - MegaLinter key: [REPOSITORY_TRUFFLEHOG]
 - Rules config: identified by [trufflehog]
 ##[endgroup]
 Unable to get number of errors with regex_count and [0-9]+:[0-9]+.*error

To Reproduce
I think any success result, but you can try my working branch.

Expected behavior
Expected no tries of getting number of errors :)

Screenshots

megalinter-out

@alexanderbazhenoff alexanderbazhenoff added the bug Something isn't working label Mar 23, 2024
@echoix
Copy link
Collaborator

echoix commented Mar 23, 2024

I see how it can happen, but it's kinda weird. Do you know if it happened with a previous version? Did the output change, or a new text is printed breaking it?

@alexanderbazhenoff
Copy link
Contributor Author

@echoix Don't know, just moved to ML from SL.

@echoix
Copy link
Collaborator

echoix commented Mar 24, 2024

But just to make sure of correctly assessing the importance here, does it happen only when there are no problems for real, and is still able to fail when the linter finds errors? (Is the green check correct ?)

@alexanderbazhenoff
Copy link
Contributor Author

alexanderbazhenoff commented Mar 24, 2024

If I knew what it depends on... I never had trufflehog errors in this repository, but next run there was no fail tries of regex.

megalinter-out2

I'll try to see when.

@alexanderbazhenoff
Copy link
Contributor Author

Well, now I am not sure that it was trufflehog. Can you see this log? Now it's after kics.

What debug variable I should set to see who's output this?

@alexanderbazhenoff alexanderbazhenoff changed the title do not get number of errors on trufflehog success Unable to get number of errors with regex_number Mar 24, 2024
@echoix
Copy link
Collaborator

echoix commented Mar 24, 2024

It isn't kicks, but more likely ansible:

cli_lint_errors_regex: ": ([0-9]+) failure\\(s\\), .* warning\\(s\\) on .* files"

I'm looking a bit, but not on how to fix your code, on why is the output not matching the regex. But at the linter reports that it is a failure. But the .nice-to-have" error count is not correct

@echoix
Copy link
Collaborator

echoix commented Mar 24, 2024

I wonder if it is ansible/ansible-lint#4007, that was fixed in ansible/ansible-lint#4026 and released in 24.2.1, the same day as our last release, so it wasn't included in yet. Ansible 24.2.1 is in our beta version though (beta is the current build of the main branch). You might want to try it, But you still have 678 fatal violations in the meantime : https://github.com/alexanderbazhenoff/ansible-collection-linux/actions/runs/8410583902/job/23029212065#step:4:1570

@echoix
Copy link
Collaborator

echoix commented Mar 24, 2024

The warning seems to be hit when preparing the output for the next tool.

@alexanderbazhenoff
Copy link
Contributor Author

alexanderbazhenoff commented Mar 24, 2024

Please check this log, there was no ansible lint.

Most of ansible-lint violation comes because these roles are made for Ansible 2.9 and linted for ansible 2.10 max (ansible-lint 5.4.0 using ansible 2.10.17). Since 2022 they add a lot of new ansible-lint rules. I need to lint and test if first on my local installed ansible-core 2.15. So I think that while I'm doing it, there's will be a new version of ML with latest ansible :) Anyway thanks for the quick reaction.

@echoix
Copy link
Collaborator

echoix commented Mar 24, 2024

Ok, that's a different one. It seems that the output format switches to GitHub Actions automatically since 2.16.0, after that format being introduced in 2.15.0 (September 2020). I think that we never caught it since the tests run inside docker, and probably the detection doesn't go through. In your only error, there isn't anything after the line:column that has the text "error", so it never matches.
Here's how they create the output:
https://github.com/adrienverge/yamllint/blob/81e9f98ffd059efe8aa9c1b1a42e5cce61b640c6/yamllint/cli.py#L80-L89

    @staticmethod
    def github(problem, filename):
        line = f'::{problem.level} file={filename},' \
               f'line={problem.line},col={problem.column}' \
               f'::{problem.line}:{problem.column} '
        if problem.rule:
            line += f'[{problem.rule}] '
        line += problem.desc
        return line

The problem level never appears after the line:column text.

The other output formats are created here:
https://github.com/adrienverge/yamllint/blob/81e9f98ffd059efe8aa9c1b1a42e5cce61b640c6/yamllint/cli.py#L49-L88

class Format:
    @staticmethod
    def parsable(problem, filename):
        return (f'{filename}:{problem.line}:{problem.column}: '
                f'[{problem.level}] {problem.message}')

    @staticmethod
    def standard(problem, filename):
        line = f'  {problem.line}:{problem.column}'
        line += max(12 - len(line), 0) * ' '
        line += problem.level
        line += max(21 - len(line), 0) * ' '
        line += problem.desc
        if problem.rule:
            line += f'  ({problem.rule})'
        return line

    @staticmethod
    def standard_color(problem, filename):
        line = f'  \033[2m{problem.line}:{problem.column}\033[0m'
        line += max(20 - len(line), 0) * ' '
        if problem.level == 'warning':
            line += f'\033[33m{problem.level}\033[0m'
        else:
            line += f'\033[31m{problem.level}\033[0m'
        line += max(38 - len(line), 0) * ' '
        line += problem.desc
        if problem.rule:
            line += f'  \033[2m({problem.rule})\033[0m'
        return line

    @staticmethod
    def github(problem, filename):
        line = f'::{problem.level} file={filename},' \
               f'line={problem.line},col={problem.column}' \
               f'::{problem.line}:{problem.column} '
        if problem.rule:
            line += f'[{problem.rule}] '
        line += problem.desc
        return line

So until we either rewrite the regex to accept all their output formats, or impose using a certain output format, you could use either the standard, colored (standard, but with colors), or parsable output format with their -f or --format cli option.
It could be set as YAML_YAMLLINT_ARGUMENTS: "--format parsable" in your config file.

Are you fluent enough in regex to suggest one that fits the multiple output formats?

@alexanderbazhenoff
Copy link
Contributor Author

alexanderbazhenoff commented Mar 24, 2024

The regex to change is:

cli_lint_errors_regex: "[0-9]+:[0-9]+.*error"

Where I can replace this file before linting in my CI? May be path inside of repo or docker, right? It's a good idea to experiment with this regex when I patch all my ten millions code violations :)

PS:

❌ Linted [REPOSITORY] files with [kics]: Found 1 error(s) - (37.01s) (expand for details)
Unable to get number of errors with regex_number and : ([0-9]+) failure\(s\), .* warning\(s\) on .* files

This time after kics... 👽

@echoix
Copy link
Collaborator

echoix commented Mar 24, 2024

The regex to change is:

cli_lint_errors_regex: "[0-9]+:[0-9]+.*error"

Where I can replace this file before linting in my CI? May be path inside of repo or docker, right? It's a good idea to experiment with this regex when I patch all my ten millions code violations :)

The way we manage to use all the linters in a standard way, while they all work differently, is through the descriptors. They are copied into the docker images built. So your approach is quite clever, since they aren't really used to build the containers, they are parsed inside. So it will work to edit them inside the containers.

The full dockerfile is here:

megalinter/Dockerfile

Lines 771 to 776 in 0909ca8

#######################################
# Copy scripts and rules to container #
#######################################
COPY megalinter/descriptors /megalinter-descriptors
COPY TEMPLATES /action/lib/.automation

But we have flavors that are trimmed down, and even individual linters published as images, like this one with just yamllint:

#######################################
# Copy scripts and rules to container #
#######################################
COPY megalinter/descriptors /megalinter-descriptors
COPY TEMPLATES /action/lib/.automation

So inside the containers, the folder containing the files to change is /megalinter-descriptors

For speeding up your debugging of just one tool (the startup and all), you could use an image with only one linter. If you run locally with mega-linter-runner (a tool that calls docker with all the arguments more easily), for this image:
https://hub.docker.com/r/oxsecurity/megalinter-only-yaml_yamllint, it would be the only-yaml_yamllint flavor.
So any of these should work

npx mega-linter-runner --flavor only-yaml_yamllint
npx mega-linter-runner --flavor only-yaml_yamllint --release beta
npx mega-linter-runner --image 'oxsecurity/megalinter-only-yaml_yamllint:beta'
npx mega-linter-runner --image 'ghcr.io/oxsecurity/megalinter-only-yaml_yamllint:beta'
npx mega-linter-runner --image 'ghcr.io/oxsecurity/megalinter-only-yaml_yamllint:beta_20240319-2208'

PS:

❌ Linted [REPOSITORY] files with [kics]: Found 1 error(s) - (37.01s) (expand for details)
Unable to get number of errors with regex_number and : ([0-9]+) failure\(s\), .* warning\(s\) on .* files

This time after kics... 👽

It's not after kicks, it's before ansible-lint. The error occurs when building the summary output, so the extra warning will be before the tool. So the problem is ansible-lint again.

Copy link
Contributor

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.

@github-actions github-actions bot added the O: stale 🤖 This issue or pull request is stale, it will be closed if there is no activity label Apr 24, 2024
@alexanderbazhenoff
Copy link
Contributor Author

Sorry, not enough time to see this. I'll be back a bit later.

@github-actions github-actions bot removed the O: stale 🤖 This issue or pull request is stale, it will be closed if there is no activity label Apr 26, 2024
Copy link
Contributor

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.

@github-actions github-actions bot added the O: stale 🤖 This issue or pull request is stale, it will be closed if there is no activity label May 26, 2024
@echoix echoix removed the O: stale 🤖 This issue or pull request is stale, it will be closed if there is no activity label Jun 4, 2024
@echoix echoix closed this as not planned Won't fix, can't repro, duplicate, stale Jun 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants