Adding VCR and WebMock to a legacy codebase

I am working on a Ruby on Rails codebase that is quite long lived, though new to me, and has an RSpec test suite that takes a long time to complete. One common piece of advice when looking to speed up the test suite in CI is to avoid out going network requests. These requests not only slow down tests by the time taken, but also make the test results dependant on an external service. I was unsure if this was an issue we faced, so I decided to investigate.

Blocking network calls with WebMock

I took the first step of using WebMock to block all network requests. First adding WebMock to the test group, by running:

bundle add webmock --group "test"

Then creating a support file spec/support/webmock.rb.

require "webmock/rspec"

WebMock.disable_net_connect!(allow_localhost: true)

Then I ran the entire test suite to understand the how many requests were being made. Every expectation that makes a network request was blocked, resulted in a forced failure. I won’t go into details, but it was definitely a problem for us with more than a few specs triggering network requests.

Setting up VCR

The next step was to add VCR to allow us to record network requests in a format that can be played back each time the test is run in the future.

bundle add vcr --group "test"

After adding the gem, I created another support file spec/support/vcr.rb.

require "vcr"

VCR.configure do |c|
  c.cassette_library_dir = "spec/vcr_cassettes"
  c.configure_rspec_metadata!
  c.hook_into :webmock
  c.allow_http_connections_when_no_cassette = true
end

Here we configure VCR where to store the recorded network requests, a.k.a the cassettes, with cassette_library_dir. The configure_rspec_metadata! option will use the spec filename, ‘describe’, ‘context’, and ‘it’ descriptions to generate record’s directories and filenames automatically. Passing :webmock to hook_into configures VCR to work with WebMock, allowing VCR to bypass it and make network requests as needed. Finally, allow_http_connections_when_no_cassette, when set to true, will allow VCR make a network call and record it as a cassette if one does not already exist.

Working together nicely

As VCR is configured to hook into and bypass WebMock, we only want to enable VCR for test expectations that we intend to make network requests. So I added the following to spec/support/vcr.rb.

RSpec.configure do |config|
  config.around :each do |example|
    if example.metadata[:vcr]
      example.run
    else
      VCR.turned_off { example.run }
    end
  end
end

Now that it is all configured, any expectation that makes a network request will be blocked by WebMock and fail by default. If the network call is expected, then the we can tag the expectation with :vcr, then run the spec to record the network call and create a cassette to be used in future.

it "calls external service A", :vcr do
  # This code is affected by VCR
  # If there is a cassette not a network request will happen
  # Otherwise a network request will be made and recorded
end

it "calls external service B" do
  # This is not affected by VCR
  # any network request will be blocked by WebMock
end

Adding to a legacy codebase

When adding VCR and WebMock to a legacy codebase, it’s not just a simple case of identifying expectations that make network calls, adding the :vcr tag, and calling it day. Each expectation may need a bit more of an investigation and taking care of everything in one PR would be time consuming to both complete and review. Instead we can add the following block to spec/support/webmock.rb.

RSpec.configure do |config|
  config.around(disable_webmock: true) do |example|
    WebMock.disable!
    example.run
    WebMock.enable!
  end
end

We can then add the :disable_webmock tag to any spec any legacy spec that we are not ready to prevent from making external calls.

RSpec.describe Services::Foo, :disable_webmock, type: :service do
  # legacy specs that trigger network calls
end

This allows the spec to run as before without being blocked from making network calls, but prevents any news specs without the tag from doing the same. Each legacy spec can be tackled in their own PR by removing the :disable_webmock tag, investigating the network requests, then refactoring or using vcr as required.

Summing up

I find this incremental introduction of a tool very helpful when working with a legacy codebase. We can get benefits for any new code right away without having to create an expansive PR coving everything that may also potentially other developers. Once the initial setup is merged, each problem spec can then be tackled on it’s own concise PR.