Troubleshooting Plugin Memory Exhaustion

To ensure a well-behaved and stable system, Quality runs analysis of your code with strict memory and time limits. By default, plugins (as of September 2018) run with a memory limit of 1 GB on Code Climate: Enterprise.

Some plugins, such as our own duplication plugin which supplies two of Code Climate's built-in maintainability checks, are more memory intensive than others. With large repositories, these plugins can hit their configured memory threshold and consequently error out. Here's an example of the duplication plugin throwing a resource exhaustion error:

/usr/src/app/lib/cc/engine/analyzers/file_thread_pool.rb:61:in `initialize': can't create Thread: Resource temporarily unavailable (ThreadError)
    from /usr/src/app/lib/cc/engine/analyzers/file_thread_pool.rb:61:in `new'
    from /usr/src/app/lib/cc/engine/analyzers/file_thread_pool.rb:61:in `with_thread_abort_on_exceptions'
    from /usr/src/app/lib/cc/engine/analyzers/file_thread_pool.rb:22:in `block in run'
    from /usr/src/app/lib/cc/engine/analyzers/file_thread_pool.rb:21:in `initialize'
    from /usr/src/app/lib/cc/engine/analyzers/file_thread_pool.rb:21:in `new'
    from /usr/src/app/lib/cc/engine/analyzers/file_thread_pool.rb:21:in `run'
    from /usr/src/app/lib/cc/engine/analyzers/reporter.rb:72:in `process_files'
    from /usr/src/app/lib/cc/engine/analyzers/reporter.rb:23:in `run'
    from /usr/src/app/lib/cc/engine/duplication.rb:59:in `block (2 levels) in run'
    from /usr/src/app/lib/cc/engine/duplication.rb:49:in `each'
    from /usr/src/app/lib/cc/engine/duplication.rb:49:in `block in run'
    from /usr/src/app/lib/cc/engine/duplication.rb:48:in `chdir'
    from /usr/src/app/lib/cc/engine/duplication.rb:48:in `run'
    from /usr/src/app/bin/duplication:28:in `<main>'

Below is an explanation of how to analyze, reduce and adjust memory thresholds for plugins. Note that memory thresholds are only adjustable by customers in Code Climate: Enterprise.

🚧

Caveat Emptor

High memory thresholds can cause issues for example in poorly provisioned or highly concurrent environments.

We recommend checking with our team before updating any plugin memory limits.

Check your exclusion paths

Most repositories contain a number of files that aren't intended for analysis and can unnecessarily bloat memory usage. For example, autogenerated files, test files, and 3rd party libraries should be excluded. See our documentation on excluding files and folders for details how to add various kinds of exclusions.

Understand memory consumption

While you can adjust memory limits on Code Climate: Enterprise, you may find it easier to monitor docker consumption locally before committing to any particular limit. The following instructions will show you how to run Code Climate locally to isolate and measure memory usage a plugin.

1. Install the Code Climate CLI

2. Install the plugins:

Ensure your Code Climate plugins are installed by running the following in the root of your repository:

$ codeclimate engines:install

3. Run your analysis locally

Restrict the run to the plugin experiencing memory issues. For example, to run only duplication, run:

$ codeclimate analyze -e duplication

4. Experiment with memory limits and monitor memory usage:

The environment variable ENGINE_MEMORY_LIMIT_BYTES sets an upper bound on the amount of memory a plugin can use. To use an upper limit of 8 GB for example, run:

$ ENGINE_MEMORY_LIMIT_BYTES=8000000000 codeclimate analyze -e duplication

You can monitor memory usage (as well as see the upper bound that was set), using docker stats. For example, again with duplication:

$ docker stats $(docker ps | grep duplication | awk '{print $1}')

Adjust the memory limit on the repository experiencing the issue

Super admins (see: Code Climate Glossary) in Code Climate: Enterprise have the ability to adjust memory thresholds and timeouts of a plugin run. Note that the thresholds apply to a plugin run (not the total build), and the same settings will be applied to every plugin in the build.

To adjust the memory limit:

1. Navigate to your Site Admin by clicking the gear in the upper right hand corner of your dashboard and selecting "Site Admin"
2. Navigate to your repository by selecting an organization and then a repository.
3. On the bottom of the page, you should see a section entitled "Engine Limits":

4. Click edit, put in a value into Memory (in bytes) and click Save.