Advanced Topics

This section covers advanced configuration and usage patterns for solidity-coverage.

Skipping Tests

Sometimes it's convenient to skip specific tests when running coverage, especially if they are gas-intensive or designed for a live network. You can do this by tagging your test descriptions and setting appropriate filters in the .solcover.js mocha options.

Example:

First, tag a test you want to skip in its description:

// Mocha test to skip
it("is a gas usage simulation [ @skip-on-coverage ]", async function(){
  // ... test logic
})

Then, configure .solcover.js to exclude tests with that tag using grep and invert:

// .solcover.js
module.exports = {
  mocha: {
    grep: "@skip-on-coverage", // Find everything with this tag
    invert: true               // Run the inverse of the matched set.
  }
};

Workflow Hooks

The plugin exposes a set of workflow hooks that let you run arbitrary async logic between the main stages of the coverage generation process. These are useful for tasks like launching secondary services (e.g., an Oraclize/Provable bridge) or performing special preparatory steps before your tests run.

The hooks are executed in the following order:

Stage	Hook Name
After instrumentation, before compile.	onPreCompile
After contract compilation.	onCompileComplete
After server launch, before tests.	onServerReady
After tests complete, before reporting.	onTestsComplete
After Istanbul reports are generated.	onIstanbulComplete

Each hook is an async function that receives a config object with relevant path and network info.

Example:

// .solcover.js
const { provableBridge } = require('./helpers');

async function serverReadyHandler(config){
  await provableBridge(config.port);
}

module.exports = {
  onServerReady: serverReadyHandler,
};

Reducing the Instrumentation Footprint

For very large projects or gas-sensitive logic, it can be useful to minimize the amount of code injected by the instrumentation process. solidity-coverage tracks four metrics: lines, branches, functions, and statements.

Disabling statement and function coverage can improve performance and reduce the likelihood of hitting solc compiler limits.

// .solcover.js
module.exports = {
  measureStatementCoverage: false,
  measureFunctionCoverage: false
};

Generating a Test Matrix

solidity-coverage can generate a "test matrix," a JSON file that maps each line of your Solidity code to the specific tests that execute it. This is useful for advanced testing strategies like mutation testing or fault localization.

To generate the matrix, run the coverage task with the --matrix flag:

npx hardhat coverage --matrix

This will create two files in your project root:

• testMatrix.json: The mapping of source code lines to tests.
• mochaOutput.json: Test run data, similar to Mocha's built-in JSON reporter.

Example Test Matrix Output

Below is an example of the testMatrix.json format:

{
  "contracts/EtherRouter/EtherRouter.sol": {
    "23": [
      {
        "title": "Resolves methods routed through an EtherRouter proxy",
        "file": "test/etherrouter.js"
      }
    ],
    "42": [
      {
        "title": "Resolves methods routed through an EtherRouter proxy",
        "file": "test/etherrouter.js"
      }
    ]
  },
  "contracts/MetaCoin.sol": {
    "16": [
      {
        "title": "should put 10000 MetaCoin in the first account",
        "file": "test/metacoin.js"
      },
      {
        "title": "should send coin correctly",
        "file": "test/metacoin.js"
      }
    ]
  }
}

Parallelization in CI

While solidity-coverage does not work with Hardhat's built-in parallel mode, you can parallelize coverage runs in CI environments. The general approach is:

1. Partition your test files into several groups.
2. Run multiple concurrent CI jobs, each executing the coverage task on a different partition of test files using the --testfiles flag.
3. Cache and combine the coverage.json output from each job into a final report. Tools like istanbul-combine-updated can be used for this step.

Note: If you use a service like Codecov, it will often automatically combine coverage reports uploaded from multiple jobs, simplifying this process.

Coverage Threshold Checks

Istanbul, a dependency of solidity-coverage, includes a command-line utility to enforce coverage thresholds. You can add a script to your package.json to check if your coverage meets a minimum percentage, causing your CI pipeline to fail if it drops below the threshold.

Example command:

npx istanbul check-coverage ./coverage.json --statements 99 --branches 94 --functions 99 --lines 99

This command will exit with an error if any of the specified coverage metrics are below the provided percentages.