Support for test coverage diff report

[arturmatos]

Didn't find anything like this either as an old issue or as a plugin so I decided to post it here. Perfectly happy to develop it myself as a plugin (assuming it is feasible).

I would like to have a imhotep plugin that adds a coverage difference report to PRs as a comment on top. Something like this:

Filename          Stmts    Miss    Cover     Missing
----------------  -------  ------  --------  ---------
dummy/dummy.py    -        -2      +50.00%   -2, -5
dummy/dummy2.py   +2       -       +100.00%
TOTAL             +2       -2      +50.00%

I think this will be very helpful to see test coverage improvements with each PR.

There are several tools for this including pycobertura and diffcover. Assuming pycobertura I would need to do this:

• Check master
• Check code under the given PR
• Run pytest with coverage on for master
• Run pytest with coverage on for the given PR.
• Run pycobertura diff coverage.old.xml coverage.new.xml --source1 old_source/ --source2 new_source/. This will generate a text report
• Return that from imhotep invoke(). The output would have to be for the whole PR and not for specific lines. Does imhotep supports this?

Would this be something sensible? Happy to develop it as a separate plugin but I'll need some guidance.

[justinabrahms]

Sorry for my delay. My memory of the imhotep code base is that it runs commands on a project basis, but aggregates results per file. You would need to have a plug-in that runs on all of the files and accumulates state as it goes. I don't know how feasible this is and it's unfortunately not something I can help you with given my work situation. Sorry. :-/

…

On Tue, Mar 6, 2018, 11:17 AM arturmatos ***@***.***> wrote: Didn't find anything like this either as an old issue or as a plugin so I decided to post it here. Perfectly happy to develop it myself as a plugin (assuming it is feasible). I would like to have a imhotep plugin that adds a coverage difference report to PRs as a comment on top. Something like this: Filename Stmts Miss Cover Missing ---------------- ------- ------ -------- --------- dummy/dummy.py - -2 +50.00% -2, -5 dummy/dummy2.py +2 - +100.00% TOTAL +2 -2 +50.00% I think this will be very helpful to see test coverage improvements with each PR. There are several tools for this including pycobertura and diffcover. Assuming pycobertura I would need to do this: - Check master - Check code under the given PR - Run pytest with coverage on for master - Run pytest with coverage on for the given PR. - Run pycobertura diff coverage.old.xml coverage.new.xml --source1 old_source/ --source2 new_source/. This will generate a text report - Return that from imhotep invoke(). The output would have to be for the whole PR and not for specific lines. Does imhotep supports this? Would this be something sensible? Happy to develop it as a separate plugin but I'll need some guidance. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#103>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAPDUzR4RJ05FCd20-5u-Q3Od-A_DOEks5tbraOgaJpZM4Se_kR> .