🥘 Kettle::Soup::Cover

---

🌻 Synopsis

Four lines of code to get a configured, curated, opinionated, set of dependencies for Test Coverage, and that's including the two lines for require "simplecov", and SimpleCov.start.

Configured for what? To work out of the box on every CI*. Batteries included. For apps and libraries. Any test framework. Many code coverage related GitHub Actions (example configs 1, 2).

| Test Framework | Helper | Config | |----------------|-----------------------------|-------------------------------| | MiniTest | test helper | .simplecov | | RSpec | spec helper | .simplecov |

📔 DO YOU LIKE PATTERNS?!? 📔

This library's local dev / testing / CI dependency structure serves as an example of a "modular gemfile" pattern enabling a discrete gemfile for each CI workflow.

What is a modular gemfile?

This modular pattern has the following benefits: - All dependencies are DRY, never repeated. - All modular gemfiles are shared between the main `Gemfile`, and the workflow `gemfiles/*.gemfile`s that need them. - All gemfiles source from the `gemspec`. For another example of this pattern see the [kettle-soup-cover](https://github.com/kettle-rb/kettle-soup-cover) gem. If you like this idea, there is an even better alternative. I've codified it for reuse in my [appraisal2](https://github.com/appraisal-rb/appraisal2/) gem, which is a hard fork of the venerable `appraisal` gem due to that gem's lack of support for modular gemfiles.

📔 ME TOO! 📔

One of the major benefits of using this library is not having to figure out how to get multiple coverage output formats working. I did that for you, and I got all of them working, at the same time together, or al la carte. Kum-ba-ya.

A quick shot of 12-factor coverage power, straight to your brain:

console export K_SOUP_COV_DO=true # Means you want code coverage export K_SOUP_COV_FORMATTERS="html,tty" # Set to some slice of "html,xml,rcov,lcov,json,tty" export K_SOUP_COV_MIN_BRANCH=53 # Means you want to enforce X% branch coverage export K_SOUP_COV_MIN_HARD=true # Means you want the build to fail if the coverage thresholds are not met export K_SOUP_COV_MIN_LINE=69 # Means you want to enforce X% line coverage export MAX_ROWS=5 # Setting for simplecov-console gem for tty output, limits to the worst N rows of bad coverage

I hope I've piqued your interest enough to give it a ⭐️ if the forge you are on supports it.

What does the name mean?

A Covered Kettle of SOUP (Software of Unknown Provenance) The name is derived in part from the medical devices field, where this library is considered a package of [SOUP](https://en.wikipedia.org/wiki/Software_of_unknown_pedigree).

💡 Info you can shake a stick at

Federated DVCS

Find this repo on other forges

| Federated [DVCS][💎d-in-dvcs] Repository | Status | Issues | PRs | Wiki | CI | Discussions | |--------------------------------------------------------|-------------------------------------------------------------------|---------------------------|--------------------------|---------------------------|--------------------------|------------------------------| | 🧪 [kettle-rb/kettle-soup-cover on GitLab][📜src-gl] | The Truth | [💚][🤝gl-issues] | [💚][🤝gl-pulls] | [💚][📜wiki] | 🏀 Tiny Matrix | ➖ | | 🧊 [kettle-rb/kettle-soup-cover on CodeBerg][📜src-cb] | An Ethical Mirror ([Donate][🤝cb-donate]) | [💚][🤝cb-issues] | [💚][🤝cb-pulls] | ➖ | ⭕️ No Matrix | ➖ | | 🐙 [kettle-rb/kettle-soup-cover on GitHub][📜src-gh] | A Dirty Mirror | [💚][🤝gh-issues] | [💚][🤝gh-pulls] | ➖ | 💯 Full Matrix | [💚][gh-discussions] | | 🎮️ [Discord Server][✉️discord-invite] | [![Live Chat on Discord][✉️discord-invite-img]][✉️discord-invite] | [Let's][✉️discord-invite] | [talk][✉️discord-invite] | [about][✉️discord-invite] | [this][✉️discord-invite] | [library!][✉️discord-invite] |

Enterprise Support

Need enterprise-level guarantees?

[![Get help from me on Tidelift][🏙️entsup-tidelift-img]][🏙️entsup-tidelift] - 💡Subscribe for support guarantees covering _all_ FLOSS dependencies - 💡Tidelift is part of [Sonar][🏙️entsup-tidelift-sonar] - 💡Tidelift pays maintainers to maintain the software you depend on!
📊`@`Pointy Haired Boss: An [enterprise support][🏙️entsup-tidelift] subscription is "[never gonna let you down][🧮kloc]", and *supports* open source maintainers Alternatively: - [![Live Chat on Discord][✉️discord-invite-img]][✉️discord-invite] - [![Get help from me on Upwork][👨🏼‍🏫expsup-upwork-img]][👨🏼‍🏫expsup-upwork] - [![Get help from me on Codementor][👨🏼‍🏫expsup-codementor-img]][👨🏼‍🏫expsup-codementor]

| Tokens to Remember | | |-------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Works with MRI Ruby 3 | | | Works with MRI Ruby 2 | | | Source | | | Documentation | | | Compliance | | | Style | | | Support | | | Maintainer 🎖️ | | | ... 💖 | 🧊 🐙 🛖 🧪 |

Format / Library x CI Matrix

This tool leverages other tools to make hard things easier, but sometimes those other tools break... I'll try to track that here.

| Format | Library | Status | Web | Circle
CI | Git
Lab | Travis
CI | Jenkins
X | Jenkins | Hudson | Semaphore | Bit
Bucket | Team
City | 🤓
Nerds | |--------|----------------------------|--------|-----|---------------|-------------|---------------|---------------|---------|--------|-----------|----------------|---------------|--------------| | html | simplecov-html | ✅ | ✅ | | | | | | | | | | ✅ | | xml | simplecov-cobertura | ✅ | | | ✅ | | | ✅ | | | | | ✅ | | rcov | simplecov-rcov | ✅ | | | | | | | ✅ | | | | ✅ | | lcov | simplecov-lcov | ✅ | | ✅ | | ✅ | ✅ | | | ✅ | | ✅ | ✅ | | json | simplecov_json_formatter | ✅ | | ✅ | | ✅ | ✅ | | | | ✅ | | ✅ | | tty | simplecov-console | ✅ | | | | | | | | | | | ✅ |

If you find this working/not working different than above please open an issue / PR!

☝️ Not actually every CI

This gem does not add coverage parsing to CI's that don't have it, since that's impossible. Vendor-specific formats which are not shared by other vendors are also not supported (e.g. BuildKite).

You'll have to configure them manually if you use them:

• BuildKite's custom simplecov extension
• GitHub Actions doesn't parse test output, but...
  • I configure my coverage workflow (see example) to upload coverage reports to SaaS services like:
  • codecov.io (has tokenless OIDC option!)
  • QLTY.sh (needs token for upload)
  • coveralls.io
  • This gem helps me configure my coverage workflow to use Github Actions designed to report coverage like:
  • Repo: irongut/CodeCoverageSummary
  • Repo: marocchino/sticky-pull-request-comment

This library is based on ideas I originally introduced in the gem [rspec-stubbedenv](https://github.com/pboling/rspec-stubbedenv).

✨ Installation

Install the gem and add to the application's Gemfile by executing:

$ bundle add kettle-soup-cover

If bundler is not being used to manage dependencies, install the gem by executing:

$ gem install kettle-soup-cover

🔒 Secure Installation

For Medium or High Security Installations

This gem is cryptographically signed, and has verifiable [SHA-256 and SHA-512][💎SHA_checksums] checksums by [stone_checksums][💎stone_checksums]. Be sure the gem you install hasn’t been tampered with by following the instructions below. Add my public key (if you haven’t already, expires 2045-04-29) as a trusted certificate: ```console gem cert --add <(curl -Ls https://raw.github.com/galtzo-floss/certs/main/pboling.pem) ``` You only need to do that once. Then proceed to install with: ```console gem install kettle-soup-cover -P MediumSecurity ``` The `MediumSecurity` trust profile will verify signed gems, but allow the installation of unsigned dependencies. If you want to up your security game full-time: ```console bundle config set --global trust-policy MediumSecurity ``` `MediumSecurity` instead of `HighSecurity` is necessary if not all the gems you use are signed. NOTE: Be prepared to track down certs for signed gems and add them the same way you added mine.

🔧 Basic Usage

RSpec or MiniTest

In your spec/spec_helper.rb or tests/test_helper.rb, just before loading the library under test, add two lines of code:

With Ruby 2.7+

``ruby require "kettle-soup-cover" require "simplecov" if Kettle::Soup::Cover::DO_COV #.simplecov` is run here!

IMPORTANT: If you are using MiniTest instead of RSpec, also do this (and not in .simplecov):

SimpleCov.externalatexit = true

```

Example: Rails & RSpec

In your spec/rails_helper.rb

```ruby

External gems

require "kettle-soup-cover"

This file is copied to spec/ when you run 'rails generate rspec:install'

We provide a preconfigured version compatible with Rails 8

require "spechelper" ENV["RAILSENV"] ||= "test"

Last thing before loading the app-under-test is code coverage.

require "simplecov" if Kettle::Soup::Cover::DOCOV # .simplecov is run here! require File.expandpath("../config/environment", dir) ```

P.S. Ensure that you have require: false on the gem in the Gemfile, and that it is in both :development and :test groups, since it ships a coverage rake task:

ruby group :development, :test do gem "kettle-soup-cover", "~> 1.0", ">= 1.0.10", require: false end

Projects that run tests against older Ruby versions, e.g. with Appraisals

```ruby

NOTE: Gemfiles for older rubies won't have kettle-soup-cover.

The rescue LoadError handles that scenario.

begin require "kettle-soup-cover"

if Kettle::Soup::Cover::DO_COV require "simplecov" # .simplecov is run here!

# IMPORTANT: If you are using MiniTest instead of RSpec, also do this (and not in .simplecov):
# SimpleCov.external_at_exit = true

end rescue LoadError => error # check the error message, if you are so inclined, and re-raise if not what is expected raise error unless error.message.include?("kettle") end ```

All projects

In your .simplecov file, add 2 lines of code:

```ruby require "kettle/soup/cover/config" # 12-factor, ENV-based configuration, with good defaults!

you could do this somewhere else, up to you, but you do have to do it somewhere

SimpleCov.start ```

See Advanced Usage below for more info, but the simplest thing is to run all the coverage things, which is configured by default on CI. To replicate that locally you could:

console CI=true bundle exec rake test # or whatever command you run for tests.

That's it!

Rakefile

You'll need to have your test task defined. If you use spec instead, you can make it a pre-requisite of the test task with:

ruby desc "run spec task with test task" task test: :spec

This gem provides a coverage task. It runs the test task (see just above about that), and opens the coverage results in a browser.

ruby require "kettle-soup-cover" Kettle::Soup::Cover.install_tasks

Filters

There are two built-in SimpleCov filters which can be loaded via Kettle::Soup::Cover.load_filters.

You could use them like this: ruby SimpleCov.add_group("Too Long", Kettle::Soup::Cover::Filters::GtLineFilter.new(1000))

Advanced Usage

There are a number of ENV variables that control things within this gem. All of them can be found, along with their default values, in lib/kettle/soup/cover.rb.

Handy List of ENV Variables

Most are self explanatory. I tried to follow POLS, the principle of least surprise, so they mostly DWTFYT. Want to help improve this documentation? PRs are easy!

console K_SOUP_COV_COMMAND_NAME K_SOUP_COV_DEBUG K_SOUP_COV_DIR K_SOUP_COV_DO K_SOUP_COV_FILTER_DIRS K_SOUP_COV_FORMATTERS K_SOUP_COV_MERGE_TIMEOUT K_SOUP_COV_MIN_BRANCH K_SOUP_COV_MIN_HARD K_SOUP_COV_MIN_LINE K_SOUP_COV_MULTI_FORMATTERS K_SOUP_COV_PREFIX K_SOUP_COV_OPEN_BIN K_SOUP_COV_USE_MERGING K_SOUP_COV_VERBOSE

Additionally, some of the included gems, like simplecov-console, have their own complete suite of ENV variables you can configure.

Compatible with GitHub Actions for Code Coverage feedback in pull requests

If you don't want to configure a SaaS service to update your pull requests with code coverage there are alternatives.

After the step that runs your test suite use one or more of the following.

irongut/CodeCoverageSummary

Repo: irongut/CodeCoverageSummary

```yaml

  - name: Code Coverage Summary Report
    uses: irongut/CodeCoverageSummary@v1.3.0
    if: ${{ github.event_name == 'pull_request' }}
    with:
      filename: ./coverage/coverage.xml
      badge: true
      fail_below_min: true
      format: markdown
      hide_branch_rate: false
      hide_complexity: true
      indicators: true
      output: both
      thresholds: '100 100' # '<MIN LINE COVERAGE> <MIN BRANCH COVERAGE>'
    continue-on-error: ${{ matrix.experimental != 'false' }}

```

marocchino/sticky-pull-request-comment

Repo: marocchino/sticky-pull-request-comment

yaml - name: Add Coverage PR Comment uses: marocchino/sticky-pull-request-comment@v2 if: ${{ github.event_name == 'pull_request' }} with: recreate: true path: code-coverage-results.md continue-on-error: ${{ matrix.experimental != 'false' }}

🔐 Security

See SECURITY.md.

🤝 Contributing

If you need some ideas of where to help, you could work on adding more code coverage, or if it is already 💯 (see below), check the REEK list, GitHub issues, GitLab issues, CodeBerg issues, or use the gem and think about how it could be better.

We so if you make changes, remember to update it.

See CONTRIBUTING.md for more detailed instructions.

🚀 Release Instructions

See CONTRIBUTING.md.

Code Coverage

🪇 Code of Conduct

Everyone interacting with this project's codebases, issue trackers, chat rooms and mailing lists agrees to follow the .

🌈 Contributors

Made with contributors-img.

Also see GitLab Contributors: https://gitlab.com/kettle-rb/kettle-soup-cover/-/graphs/main

⭐️ Star History

📌 Versioning

This Library adheres to . Violations of this scheme should be reported as bugs. Specifically, if a minor or patch version is released that breaks backward compatibility, a new version should be immediately released that restores compatibility. Breaking changes to the public API will only be introduced with new major versions.

📌 Is "Platform Support" part of the public API?

Yes. But I'm obligated to include notes...

SemVer should, but doesn't explicitly, say that dropping support for specific Platforms is a breaking change to an API. It is obvious to many, but not all, and since the spec is silent, the bike shedding is endless.

dropping support for a platform is both obviously and objectively a breaking change

• Jordan Harband (@ljharb, maintainer of SemVer) in SemVer issue 716

To get a better understanding of how SemVer is intended to work over a project's lifetime, read this article from the creator of SemVer:

• "Major Version Numbers are Not Sacred"

As a result of this policy, and the interpretive lens used by the maintainer, you can (and should) specify a dependency on these libraries using the Pessimistic Version Constraint with two digits of precision.

For example:

ruby spec.add_dependency("kettle-soup-cover", "~> 1.0")

See CHANGELOG.md for a list of releases.

📄 License

The gem is available as open source under the terms of the MIT License . See LICENSE.txt for the official Copyright Notice.

© Copyright

• Copyright (c) 2023-2025 Peter H. Boling, of Galtzo.com , and kettle-soup-cover contributors

🤑 One more thing

Having arrived at the bottom of the page, please endure a final supplication. The primary maintainer of this gem, Peter Boling, wants Ruby to be a great place for people to solve problems, big and small. Please consider supporting his efforts via the giant yellow link below, or one of the smaller ones, depending on button size preference.

P.S. If you need help️ or want to say thanks, 👇 Join the Discord.