Our new test reporter, documented below, is in Public Beta, and subject to breaking changes in the future.
If you prefer not to use our Public Beta, you are welcome to (continue to) use our soon to be deprecated older test reporters.
The Code Climate test coverage reporter takes a supported test coverage report, transforms it into a generalized format, and submits it to the Code Climate API. It can also combine multiple test coverage report payloads created from parallel test processes within a parallelized CI build.
Code Climate uses test coverage reports to provide test coverage annotations on codeclimate.com and via the GitHub browser extension. If configured, Code Climate will also send test coverage changes to enabled integrations.
Code Climate's test reporter relies on third party (typically open source) tools to generate local test coverage reports.
In order to use the Code Climate test reporter, please first ensure you're able to generate test coverage in a supported format locally.
If you're having issues generating local coverage, please contact the maintainers of the test coverage library you're using (e.g. SimpleCov, Instanbul)
The test reporter currently supports the following test coverage languages and formats:
More languages and formats are coming soon.
You can fetch a pre-built binary from the following URLs:
If you'd like to pin to a specific version of the test reporter, you should refer to the releases on GitHub.
Before using the test reporter, you should visit your Code Climate repo settings page to fetch the repo's test reporter ID. This value is required by the test reporter to correctly identify which repo the test report should be submitted for.
Before running your test suite, run the
before-build command to notify Code Climate of a pending report.
After your test suite completes, run the
after-build command to send a formatted report or notify Code Climate of a failing build. Some CI providers expose the exit code as an environment variable. Otherwise, if you're executing the test reporter within the same step, you may use
$? to pass the last exit code to the reporter.
cc-test-reporter after-build --exit-code $?
Code Climate assists with coverage for parallel testing and multiple suites by offering lower level commands to join together partial test coverage results and then post the final result to Code Climate.
In cases where test suites are being run across multiple VMs, you will also need upload partial test coverage data after a batch of tests and fetch all partial test coverage data after all batches complete. While our reporter does not handle this for you, there are a variety of relatively simple options for performing this. We recommend syncing and fetching files from S3 as one approach.
See the README on GitHub for additional instructions.
We support any CI provider
Below, you'll find an examples of how to configure our test reporter with some common CI providers.
For more examples, check out the test reporter repo on Github.
If you run into an issue, follow the steps outlined below:
- If there's an issue with the reporting (or with Code Climate), remove Code Climate from the equation.
- Look at coverage results locally. Are they what you expect?
- Are your results what you expect, regardless of Code Climate? How are they different while looking at Code Climate?
For more detailed setup instructions, please visit the test reporter repo on Github.
To assist with troubleshooting, use the
--debug flag to output debug messages during operation.
For more information on common errors with our test reporter, refer to our doc on Troubleshooting Test Coverage.
Q: When Code Climate is down, it causes my builds to fail because I can't upload my coverage reports. How can I fix this?
A: We're looking to update our reporter to account for this. In the meantime, we recommend the following workarounds:
after-build || echo “Skipping CC coverage upload” or
upload-coverage || echo “Skipping CC coverage upload”
Q: Where should I put the
A: If you're running your tests in single builds, use
after-build --debug. For tests run in parallel, please use both
format-coverage --debug and
For help with the configuration of this test reporter, please contact support. To help expedite the troubleshooting process, please include the following items in your support ticket:
- Name of your repo
- Name of your CI
- Are you running tests in single or parallel builds?
- A copy of your CI configuration file
- Output from your CI (after including the
--debugflag in your CI configuration)
- A copy of your
codeclimate.json(this can be generated inside your
coveragefolder by running
- Any relevant screenshots of your setup