Add code coverage to CI

[TomNicholas]

@keewis pointed out that after moving to xarray-contrib we seem to have lost the CI. I've re-created the Azure pipeline that's supposed to run the CI tests (including pytest and black), which I have been able to re-run on master. I want to check it's working on new PRs.

[TomNicholas]

Also was there any other integrations we were using? PEP8speaks bot?

[TomNicholas]

@keewis I think renaming the package in #15 broke readthedocs too #13

[keewis]

yes, I forgot to update the import in conf.py. This is fixed in #11, so if / when we merge that, RTD builds will be fixed, too (I could create a new PR for that if necessary, though).

[keewis]

I think the only extra integration we had was pep8speaks, although I'm not quite sure how much value this has: flake8 reports more than just PEP8, and usually the reported errors are fixed by a single run of black.

Also, while we're at it: do you think we can investigate the actions integration of pipelines? Right now, only a single CI action is visible (see also #11 (comment))

[TomNicholas]

This is fixed in #11

Okay great, thanks.

I think the only extra integration we had was pep8speaks, although I'm not quite sure how much value this has

It's kind of nice that it comments on the PR says what was wrong with the code, but I agree it's not really adding that much.

do you think we can investigate the actions integration of pipelines? Right now, only a single CI action is visible

You mean how it just says "xarray-contrib.pint-xarray" instead of listing each test individually? I'm not sure how to change that but I'll have a look.

[keewis]

me neither. Let's ping @shoyer since he seems to have been the one to set this up for xarray

[TomNicholas]

I found this SO question but it doesn't actually tell you how to split them up.

Code Coverage! We should add that too - I'll try and do it now.

[shoyer]

I'm not sure why Azure looks different here than for xarray. I don't think that's anything I explicitly set.

There might be an option hidden somewhere in the Azure Pipelines settings?

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (master@4f3b0bc). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #16   +/-   ##
=========================================
  Coverage          ?   82.41%           
=========================================
  Files             ?        7           
  Lines             ?      614           
  Branches          ?        0           
=========================================
  Hits              ?      506           
  Misses            ?      108           
  Partials          ?        0           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4f3b0bc...67fef10. Read the comment docs.

[TomNicholas]

I had a look in the settings but couldn't see anything. I've emailed to commenter on that stack overflow comment who said he works at Azure because why not.

Also @shoyer these Azure pipelines are currently just set up on an account in my name, given access to this repo. Is that the way I'm supposed to be doing it? Can I give other people access too?

@keewis I think I've got codecov working now so if you want to merge this into master we should then have coverage stats.

[keewis]

Looks good to me, but before I merge: what's the difference between both coverage checks?

[keewis]

would it make sense to add these and all of the linters to requirements.txt? Or should we rather maintain a separate CI requirements file?

[keewis]

Let's do that in a separate PR.

[keewis]

since this is about CI in general: let me know if anyone wants to become a maintainer on RTD

[keewis]

I still don't understand the difference between two coverage checks, but let's merge for now.