check current expected to fail files for closed issues

[individual-it]

• there should not be any test mentioned that is not connected to an issue
  • ocis [tests-only][full-ci]Check for issue not mentioned to a test ocis#4614
  • web [tests-only][full-ci]Check for issue not mentioned to a test in expected to failure web#7678
  • reva-master [tests-only]Check for issue not mentioned to a test in expected to failure in reva-master cs3org/reva#3262
  • reva-edge [tests-only]Check for issue not mentioned to a test in expected to failure in reva-edge cs3org/reva#3263
• there should not be any closed issues mentioned in the expected to fail file.
  • remove/investigate existing closed issues (@SagarGi @kiranparajuli589 )
  • update linter to report close issues [tests-only][full-ci]Added script to report closed issues in expected failure file core#40377 a separate linter to check it

If the test still fails then there is either a problem in the test code (fix the code) or the test fails for some other reason (reorder the test)

[individual-it]

the expected-to-fail linter could also do the checks for us

[SagarGi]

List of issues that are closed but still exist in expected-to-failure:

1. No robots.txt available ocis#1314 (@SagarGi )

   • review tests linked to this issue
   • remove them from expected to fail if they pass or create another issue if necessary -> tests need adjustment [tests-only][full-ci]Adjust Robots.txt for ocis core#40238

2. Creating a new folder which is a substring of Shares leads to "Unknown Error" ocis#3033 (@SagarGi )

   • review tests linked to this issue
   • remove them from expected to fail if they pass or create another issue if necessary [tests-only][full-ci]Add Tests for creating Files and Folder When Space Shares Jail exists for core PR 40239 ocis#4284

3. Getting content of a shared file with same name returns 500  ocis#3880 (@kiranparajuli589)

   • review tests linked to this issue
   • remove them from expected to fail if they pass or create another issue if necessary [tests-only] removed closed issue #3880 from expected failures list ocis#4290

4. Use GraphAPI for creating/deleting users and groups in acceptance test ocis#3167 (@kiranparajuli589)

   • review tests linked to this issue
   • remove them from expected to fail if they pass or create another issue if necessary [tests-only] remove closed issue #3167 from the list of expected failures ocis#4292

[amrita-shrestha]

there may be still some test mentioned that is not connected to an issue
Some refactored in this PR (owncloud/ocis#4527)

[SagarGi]

Regarding to update expected-to-fail linter to report the closed issue, we have 2 script (run.sh and lint-expected-failures.sh). run.sh calls the lint-expected-failure.sh for linting and itself checks linting for expected to failures. These run in each pipiline while running tests. So may be we do not wanna lint to check for closed issue in expexted-to-failure file in each Pipiline.

A way could be adding a pipeline only to check or lint if there are any closed issue that are still in expected to failure file.

Also one cons could be after all the closed issue has been fixed it can be a pipeline that is of no use (Since it makes API request to get the state of issue.)

NOTE: I will be trying to find out if there is another way to do it

@individual-it @phil-davis @saw-jan @grgprarup

[phil-davis]

The problem with putting a check in a CI pipeline is that, if someone closes an issue in GitHub that still has expected-failures then suddenly the CI of the repo will go red. That will be really annoying for developers (and everybody).

We can make a script that parses an expected-failures file(s), finds each issue link, then does a curl API request to GitHub and finds out the status of each issue. Then it reports all issues that are closed. A script like that can be run locally whenever we like, and then we can manually decide what to do about those issues.

[SagarGi]

The problem with putting a check in a CI pipeline is that, if someone closes an issue in GitHub that still has expected-failures then suddenly the CI of the repo will go red. That will be really annoying for developers (and everybody).

We can make a script that parses an expected-failures file(s), finds each issue link, then does a curl API request to GitHub and finds out the status of each issue. Then it reports all issues that are closed. A script like that can be run locally whenever we like, and then we can manually decide what to do about those issues.

sure !!!

[SagarGi]

All the work realated to isse have been merged. So may be we can closed this @amrita-shrestha @saw-jan. Please close it.

[saw-jan]

OKey. Closing here!