The Code Climate test coverage reporter takes a supported test coverage report, transforms it into a generalized format, and submits it to the Code Climate API. It can also combine multiple test coverage report payloads created from parallel test processes within a parallelized CI build.
Code Climate uses test coverage reports to provide test coverage annotations on codeclimate.com and via the GitHub browser extension. If configured, Code Climate will also send test coverage changes to enabled integrations.
Code Climate's test reporter relies on third party (typically open source) tools to generate local test coverage reports.
You'll find these tools listed in the Supported Languages and Formats table.
When troubleshooting test coverage submission, please first ensure you're able to generate test coverage in a supported format locally, outside of Code Climate's test reporter.
If you're having issues generating local coverage, please contact the maintainers of the test coverage library you're using (e.g. SimpleCov, Istanbul).
Use the repo's test reporter ID from your Code Climate repo settings page as an environment variable to identify your repo.
Fetch the test reporter pre-built binary.
before-buildcommand before running your test suite to notify Code Climate of a pending report.
after-buildcommand after your test suite completes to send a formatted report or notify Code Climate of a failing build.
machine: environment: CC_TEST_REPORTER_ID: ABC123 dependencies: post: - curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > ./cc-test-reporter - chmod +x ./cc-test-reporter test: pre: - ./cc-test-reporter before-build post: - ./cc-test-reporter after-build --exit-code $EXIT_CODE
The Code Climate test reporter can join multiple test reports together from separate test suite processes. Uploading partial test coverage data after a batch of tests and fetching all partial test coverage data after all batches complete is a DIY process. We recommend syncing and fetching files from S3 as one approach.
Parallel test support requires the ability for you to:
- upload and download partial test coverage data to/from shared storage (ex: AWS S3),
- and to use the following sub-commands:
run commands after each batch has completed
run commands after all tests have completed
upload the single result to Code Climate
See the README on GitHub for detailed instructions.
$ ./cc-test-reporter format-coverage --output "coverage/codeclimate.$N.json" $ aws s3 sync coverage/ "s3://my-bucket/coverage/$BUILD_NUMBER" # Where: # $N should be a unique identifier for that batch of tests # $BUILD_NUMBER should be the build number provided by your CI.
$ aws s3 sync "s3://my-bucket/coverage/$BUILD_NUMBER" coverage/ $ cc-test-reporter sum-coverage --output - --parts $PARTS coverage/codeclimate.*.json | \ cc-test-reporter upload-coverage --input - # Where: # $PARTS should be the number of payloads to sum.
And, as with single payload reporting, you'll need to:
- Use your repo's test reporter ID from your Code Climate repo settings page.
- Fetch the test reporter pre-built binary.
- Run the before-build command before running your test suite to notify of a pending report.
After those steps:
- you'll use the format-coverage, sum-coverage, and upload-coverage sub commands.
- (the after-build command is not used with parallel or multi-suite tests.)
You can fetch a pre-built binary from the following URLs:
If you'd like to pin to a specific version of the test reporter, you should refer to the releases on GitHub.
We support any CI provider
Below, you'll find an examples of how to configure our test reporter with some common CI providers.
Need more help?
Opening an issue or contacting Support?
- Repo name
- CI name
- If you use single, parallel or multi-suite builds
- A copy of your CI configuration file
- Output from your CI when using the
--debugflag in your CI configuration
- Any relevant screenshots of your setup
PS: Make sure that you're able to run coverage results locally with the basic requirements, outside of Code Climate's test reporter.