Search results for "{{ search.query }}"

No results found for "{{search.query}}". 
View All Results

Configuring Test Coverage

Public Beta

Our new test reporter, documented below, is in Public Beta, and subject to breaking changes in the future.

If you prefer not to use our Public Beta, you are welcome to (continue to) use our soon to be deprecated older test reporters.

Overview

The Code Climate test coverage reporter takes a supported test coverage report, transforms it into a generalized format, and submits it to the Code Climate API. It can also combine multiple test coverage report payloads created from parallel test processes within a parallelized CI build.

Code Climate uses test coverage reports to provide test coverage annotations on codeclimate.com and via the GitHub browser extension. If configured, Code Climate will also send test coverage changes to enabled integrations.

Supported Languages and Formats

The test reporter currently supports the following test coverage languages and formats:

Language
Format / Coverage Tool

Ruby

JavaScript

lcov (generated by Instanbul e.g.)

PHP

Go

Python

Java

Swift

gcov (generated by SwiftCov e.g.)

C/C++

Scala

More languages and formats are coming soon.

Installation

If you'd like to pin to a specific version of the test reporter, you should refer to the releases on GitHub.

Usage

Before using the test reporter, you should visit your Code Climate repo settings page to fetch the repo's test reporter ID. This value is required by the test reporter to correctly identify which repo the test report should be submitted for.

Before running your test suite, run the before-build command to notify Code Climate of a pending report.

cc-test-reporter before-build

After your test suite completes, run the after-build command to send a formatted report or notify Code Climate of a failing build. Some CI providers expose the exit code as an environment variable. Otherwise, if you're executing the test reporter within the same step, you may use $? to pass the last exit code to the reporter.

cc-test-reporter after-build --exit-code $?

Parallel Tests and Multiple Test Suites

The Code Climate test reporter can join multiple test reports together from separate test suite processes. Uploading partial test coverage data after a batch of tests and fetching all partial test coverage data after all batches complete is a DIY process. We recommend syncing and fetching files from S3 as one approach.

See the README on GitHub for additional instructions.

CI Examples

We support any CI provider

Our test reporter can run on any CI provider, even if you don't see a specific example here. If you'd like help setting up the test reporter on a different CI provider, feel free to send us a message or open an issue on GitHub.

Below, you'll find an examples of how to configure our test reporter with some common CI providers.

Troubleshooting

Error: you must supply a CC_TEST_REPORTER_ID ENV variable or pass it via the --id/-r flag

The test reporter is unable to find your test reporter ID. This value either needs to configured within the environment or passed directly as a CLI argument:

export CC_TEST_REPORTER_ID=<your token>
cc-test-reporter after-build --exit-code $?
export CC_TEST_REPORTER_ID=<your token>
cc-test-reporter after-build --id <your token> --exit-code $?

Error: file not found

This error is raised with the reporter is unable to find a file referenced with the test report. Does that file exist within your git repository? Was your test suite run within a different filesystem (such as in a docker container)?

You may need to specify a prefix value. For example, if you're running the tests within a docker container and your app code is located at /usr/src/app, the reporter run outside of the docker container will not be able to find files at the same absolute path. Pass --prefix /usr/src/app to instruct the test reporter to strip the unknown base path.

Configuring Test Coverage