Documentation (Quality)Contact SupportCode Climate API
Documentation (Quality)

GitHub Pull Requests

Suggest Edits

Code Climate provides a few different ways for users to view code quality and test coverage analysis results on a pull request.

Pull Request Review Comments

Code Climate has an associated Github App that, once installed, supports two types of automated review comments: Summary Comments and Inline Issue Comments.

Summary Comments

Summary comments are comments left on the pull request summarizing the issues introduced by the pull request, as well as the test coverage changes.

There are several configuration options for how this comment should behave:

  • Delete (default): For every push to the pull request, the original comment will be deleted and a new one will be posted. This will move the summary comment to the bottom of the pull request and will create a new notification via GitHub.
  • Update: For every push to the pull request, the original comment will be updated. This will not create new notifications via GitHub.
  • Once: A summary comment should be left once when the pull request is opened. Future pushes to the pull request will not modify the comment.

To enable this functionality, please see Configuring Summary Comments.

Inline Issue Comments

Inline issue comments are pull request comments left as part of a GitHub review. Any new issues related to lines changed in the diff will be added as comments to the Pull Request. Once addressed, the comments will collapse using the native GitHub behavior.

There are two configuration options for how this comment should behave:

  • Additional (default): For every push to the pull request, if there are new issues introduced by the changes, another review will be left with comments for the new issues. This will create a new notification via GitHub.
  • Once: A review should be left the first time issues are detected. Afterward, if new pushes to the pull request introduce additional issues, they will not be left as a new review.

In addition, the maximum number of comments left per review is configurable (default: 5).

To enable this functionality, please see Configuring Inline Issue Comments.

Installing the GitHub App

To make use of automated pull request comments, first the CodeClimate GitHub app must be installed.

This integration is configured via a separate GitHub app to utilize several advantages of GitHub Apps compared to OAuth applications:

  • Comments will be left by a single bot user, rather than a user in the GitHub organization
  • Installations are tied to a GitHub organization, rather than a specific user
  • A minimal set of permissions is required

For more information about GitHub Apps, please see the GitHub documentation.

Show me how (Install GitHub App for Comments)

📘

Note

Only Admins of a GitHub organization are authorized to install and configure GitHub apps. These steps must be completed by a GitHub Admin.

  1. Navigate to the CodeClimate GitHub App. There is also a link from the repository settings page.

  1. Click "Install" and select which GitHub organization you would like to have automated pull request comments enabled for.

The app can be installed for all current and future repositories in the GitHub organization:

Or for a select list of repositories in the GitHub organization:

  1. Verify each of the installed repositories has been added on Code Climate.

Configuring Summary Comments

  1. Verify that the GitHub app has been installed for the repo and the repo has been added to Code Climate as described above.

  2. Visit the repo settings page and navigate to GitHub settings.

  3. Enable the Summary Comments feature, choose the desired strategy, and click save.

  1. The next time a pull request is opened for one of the installed repositories, a summary comment will be posted based on the selected strategy.

  2. To add or remove this functionality for other repositories in the GitHub organization, the app installation can be managed from the GitHub organization settings page where the app has been installed under Settings -> Installed GitHub Apps.

Configuring Inline Issue Comments

  1. Verify that the GitHub app has been installed for the repo and the repo has been added to Code Climate as described above.

  2. Visit the repo settings page and navigate to GitHub settings.

  3. Enable the Inline Issue Comments feature, choose the desired strategy, and click save.

  1. The next time a pull request is opened for one of the installed repositories, a summary comment will be posted based on the selected strategy.

  2. To add or remove this functionality for other repositories in the GitHub organization, the app installation can be managed from the GitHub organization settings page where the app has been installed under Settings -> Installed GitHub Apps.

Troubleshooting: PR Comments

Missing Summary and Inline Comments? If you send test coverage to Code Climate, PR comments require test coverage payloads for the PR's commits.

The Test Coverage tab on your repo's Settings page lists your payloads: make sure there are payloads for your PR's merge base and head.

Pull Request Statuses

If you have our GitHub PR integration enabled, when opening up a pull request, we'll run our 10-point maintainability assessment and post a status update in GitHub notifying you of new code quality issues that have been introduced.

Once you've setup test coverage reporting, you can use our test coverage statuses to ensure that all PRs meet your team's specific coverage standards.

In addition to our in-house maintainability checks, you can optionally enable open source plugins to run checks against your code.

Our browser extension ensures code quality and test coverage data is constantly visible throughout your development workflow, so your team can get, discuss, and act on all of this information without leaving GitHub. By clicking over to Files Changed in GitHub, you'll be able to view the static analysis and test coverage results right in-line.

Teams should use their judgment to validate and invalidate issues found by Code Climate. For these cases, Code Climate allows developers to mark any issue with one of 3 different statuses:

  • confirmed - a legitimate issue that should be addressed
  • invalid - is a false-positive
  • wontfix - a legitimate issue, but it does not need to be addressed at this time

For more information on changing issue statuses, please see this doc.

🚧

Ineligible Pull Requests

Though we hope to expand our pull request integration in the future, note that we currently do NOT support:

  • Pull requests against branches other than your default branch.
  • Pull requests from repositories not hosted on GitHub.com. Our PR integration currently does not support pull requests from SCM's such as Bitbucket or GitHub Enterprise.

Show me how (Enable PR Status Updates)

  1. From your Repo Dashboard, navigate to Repo Settings.

  1. Select the GitHub tab.

  1. Scroll down to Pull request status updates and click Install.

📘

Note

To communicate with GitHub, the oauth token of the user installing the integration will be used. To change to a new token, the integration can be disabled and re-installed.

1708
  1. Once installed, the setting will show a green check mark indicating that it is active.

  1. Make sure that the GitHub webhook has been installed. This notifies Code Climate when pull requests are opened and updated. The webhook can be installed or re-installed via the same GitHub settings page under Connections > Webhook on GitHub.

  1. To disable the integration or rotate the GitHub token being used, click Edit next to Pull request status updates. Flip the toggle to off and click Save. The integration can then be re-installed.

  1. Once the PR integration is enabled, our test coverage statuses will be enabled by default. However, you'll need to configure test coverage in order to see those statuses populate.

Updated over 1 year ago