Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Coveralls support #38

Closed
bfontaine opened this issue Feb 10, 2014 · 7 comments
Closed

Coveralls support #38

bfontaine opened this issue Feb 10, 2014 · 7 comments

Comments

@bfontaine
Copy link
Contributor

Hi,

I’m considering writing an extension/pull-request to Cloverage to add support for Coveralls, especially on Travis CI. This is an online app that gets coverage results from tools like Cloverage on CI servers and gives you an history of your project’s coverage, test it for you when someone does a pull request, and gives you a pretty badge to put in your README. It has native support for Ruby, supports other languages like Python, PHP, Go, etc but not Clojure. Fortunately, it has an HTTP API that allows anyone to submit coverage data for a project in any language.

Here is how it would work:

  1. run your tests as usual on Travis CI with Cloverage
  2. generate a JSON report with coverage data plus two strings: the service name (here it’s Travis CI) and the job id (given by Travis CI).
  3. send this report to their API
  4. ???
  5. profit!

Do you think it would make sense to add a reporter in cloverage.report that would do all this? One could then use something like lein cloverage --coveralls. The other option I see is to generate only a JSON report that would then be handled by an external script which would add necessary CI server info and send it to Coveralls. Thoughts?

Also, Coveralls doesn’t support partially covered lines, it uses only two states: covered and uncovered. For the former, it keeps track of the number of hits.

@JacekLach
Copy link
Collaborator

That would be cool; yes, cloverage.report is the place to do this.

It seems they have 3 states, actually (0 - uncovered, positive - covered, and null - not relevant). I don't think we track how many times any particular form was hit (though adding that should be simple - see https://github.com/lshift/cloverage/blob/master/cloverage/src/cloverage/coverage.clj#L29 and relevant lines in reporters); possibly reusing 1 as partial and 2 as fully covered could work?

No ideas for step 4, I'm afraid!

@bfontaine
Copy link
Contributor Author

Thank you for your feedback, I added the report on my fork (44d824b), but how do you:

  1. create sample forms in order to test the function?
  2. test the plugin locally, without pushing to Clojars?

@JacekLach
Copy link
Collaborator

  1. I'd rather not include the http communication in this library. Instead, produce the output as $outdir/coveralls.json or something like that; then it can be curl'ed to the server if needed
  2. To test the plugin, lein install locally then run CLOVERAGE_VERSION=1.0.4-SNAPSHOT lein cloverage in a test project
  3. No real test 'framework' for the report functionality yet, I'm afraid. It should be enough for now to just add "--coveralls" to https://github.com/lshift/cloverage/blob/master/cloverage/test/cloverage/test_coverage.clj#L188 to have the 'integration test' (if you can call it that) exercise this path.

@JacekLach
Copy link
Collaborator

Added in bb0ea7c

@jcf
Copy link

jcf commented Sep 6, 2014

Have you been able to update Coveralls from a private repository?

From what I can tell you have to include a repo_token in the generated JSON meaning Coveralls support is only partially implemented.

@bfontaine
Copy link
Contributor Author

No, I didn’t have this use case, but I guess that’s easy to add, it would read the repo_token from an environment variable and include it in the generated JSON. Do you want to make the pull request? I can make one if you link the relevant docs.

@jcf
Copy link

jcf commented Sep 9, 2014

Unfortunately, I've had to piece this information together from the Ruby client because it's not mentioned in the API docs.

I had hoped you'd be able to pass credentials in the API request headers, but from what I can tell the credentials have to be added to the JSON coverage report. If you omit the repo token, and I'd guess the repo token secret, you get a 422 response… not what I'd expect from an authentication failure.

I've decided not to use the service based on my experience so far.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants