How do we know a project **should** have CI?

[harlantwood]

@RichardLitt said in #25 (comment):

Travis and CI aren't needed for some repos, and they are in the list twice. Perhaps we could have a 'Travis not enabled' instead of 'Build unknown'? Would remove two columns we'd no longer need.

The two Travis colums are actually different... One is the build status, and one is whether the Travis badge is in the readme.

Better button text sounds fine, feel free to PR, preferably with similar styling to the buttons, and probably grey color.

Since, as you point out, CI is not needed on all repos, we should not turn the latter column red, if there is no CI at all. Unfortunately, this looks roughly the same as a project that has code and tests but has set up zero CI, which we would like to "mark red".

We could say if it contains code (JS, go, rust, etc) it needs CI, and mark it red if it has none. This may be a bit of work ; )

Thoughts?

Should all repos that contain code have CI?

[RichardLitt]

I think you should check for a CI Badge, and leave it at that. There should be a CI badge on all relevant repos, at this point.

[RichardLitt]

Regarding Travis and 'build unknown', Getting a better button seems harder than I expected. For now, let's just leave it as it is.

Rethinking the CI; Do the tests fail if there are no tests set up? Should we test everything?

[harlantwood]

Rethinking the CI; Do the tests fail if there are no tests set up? Should we test everything?

Probably not. We have a lot of random repos, eg: https://github.com/ipfs/POST

[RichardLitt]

Those can be cleared out by searching for 'discussion repo' badges, which I should have added to all of them.

[RichardLitt]

OK. Can we check for code in a repository using https://github.com/github/linguist, and then automatically disqualify all non-code repositories? Everything else should have CI or Travis, right?

[RichardLitt]

Or could we just search for a .travis.yml or a circle.yml and leave it up to the project heads to make sure that these exist?

[harlantwood]

OK. Can we check for code in a repository using https://github.com/github/linguist, and then automatically disqualify all non-code repositories? Everything else should have CI or Travis, right?

👍 this sounds perfect. Maybe language is in the repo object returned by Octokat?

Or could we just search for a .travis.yml or a circle.yml and leave it up to the project heads to make sure that these exist?

👎 I don't think so... part of the idea of this project is to watch all the repos and make sure they are all following our guidelines...

[RichardLitt]

Boom! It looks like that is in the language object returned by Octokat. For https://api.github.com/repos/ipfs/ipfs, it shows language: null - same for ipfs/pm, ipfs/apps. For anything with code on it, it shows the main language of code. I think we can assume that if there is a language, it should have code, and it should have CI.

We should also check in case there is a .travis.yml or circle.yml, too: for instance, ipfs/awesome-ipfs has no code, but I have travis running to make sure that links aren't broken in any PR.

[harlantwood]

Awesome. So if no CI is found, we leave the CI column blank for language=>null repos, but put a "CI missing" badge in if language=>something.