Greg DaynesGreg Daynes

Getting Started With Minitest

• What is Minitest?
• How to start testing with Minitest
  • Begin by creating your test file
  • Write your first test
  • Write an assertion
  • Running your first test
  • Fix the failing test case
  • Add another test case
• How to write tests using Spec
  • Write your first spec
  • Update test descriptions
  • Replace test methods with it statements
  • Replace variable definitions with let statements
  • Replace class calls with subject.
• Rake Task
• Finish the test suite
• In Deeper with Minitest
  • Built in assertions
  • Using Seeds to replay tests
  • Setting up and tearing down tests
  • Skipping tests
  • Stubbing
• Benchmarking Minitest
• Minitest & Rails
• References

What is Minitest?

Minitest is a complete, lightweight suite of testing utilities for Ruby.

It was created and is maintained by the Seattle Ruby community, and was accepted into Ruby core for 1.9.

Minitest provides support for TDD, BDD, Mocks/Stubs, Benchmarking, can run Capybara specs, has a spec DSL, can be easily extended however you see fit (it’s Ruby after all).

Here is a quick example of a test in Minitest

require 'minitest/autorun'

class Coffee
  attr_reader :color

  def initialize
    @color = :black
  end
end

class CoffeeTest < Minitest::Test
  def test_color
    fresh_coffee = Coffee.new
    assert cup_of_coffee.color == :black
  end
end

How to start testing with Minitest

Getting started with Minitest is amazingly easy. If you can write Ruby (even if you’ve only written a few methods) you can write Minitest.

Begin by creating your test file

To get started, create a file called coffee_test.rb

It is recommended to follow the Minitest convention by ending the file name with _test.rb. This is not a hard requirement, but will make for a good habit when adding tests for Rails or other projects that have tooling around _test.

Next, add minitest/autorun as a requirement Then create a new class CoffeeTest as a subclass of Minitest::Test

# coffee_test.rb

require 'minitest/autorun'

class CoffeeTest < Minitest::Test
end

You can add more gems, and utilities to add even more power to Minitest, but keep it simple to start.

As your test suite grows, it’s a good idea to move all the requires and setup into common place like test_helper.rb. That way you keep your tests DRY

Write your first test

Now that the boilerplate ready, add the first test method.

getting-started-with-minitest tests are just plain old Ruby methods.

Start by adding a new method starting with test_ to the class.

# coffee_test.rb

require 'minitest/autorun'

class CoffeeTest < Minitest::Test
  def test_coffee
  end
end

The test_ prefix for the method is how Minitest distinguishes Ruby test methods and other methods in the test file, (like helper methods that handle setup, or create data for use across multiple tests).

_test_ is one way to write tests in Minitest. Another way is to use a Spec DSL, which is explained later in this post._

Write an assertion

An assertion will check to see if the returned value of a block or call evaluates to true. If so, the test will pass. If not false, the test fails.

To test the opposite, blocks or calls returning false, use refute. The default Minitest assert methods have opposite refute methods that take the same params.

Now write an assertion, in this case, we’re going to test that a new instance of Coffee is still a Coffee.

# coffee_test.rb

require 'minitest/autorun'

class CoffeeTest < Minitest::Test
  def test_coffee
    assert_instance_of Coffee, Coffee.new
  end
end

This is done using the assertion assert_instance_of, which takes a class, and an obj, checking whether or not the obj is an instance of class.

Running your first test

Running a test file is the same as any Ruby script.

Start up a Terminal in the same location as your test file, and run the following command.

$ ruby coffee_test.rb

The output should be similar to the following.

Run options: --seed 35012

# Running:

E

Finished in 0.000729s, 1371.7422 runs/s, 0.0000 assertions/s.

1) Error:
CoffeeTest#test_coffee:
NameError: uninitialized constant CoffeeTest::Coffee
Did you mean?  CoffeeTest
    test_001b.rb:5:in `test_coffee'

1 runs, 0 assertions, 0 failures, 1 errors, 0 skips

Great! Sort of, the test ran successfully, but the it failed.

Fix the failing test case

Looking over the output from the test run, notice the following:

• Run options: --seed 35012 This is the seed test was run with. It is randomized each time the test is run. More on seeds below.
• A series of ...s and Fs. These signify successes and exceptions. For now it is just a single E
• How long the testing took, how many times it will run in per second, and how many assertions we can run per second. This is useful later in determining how fast your suite is. Fast tests = faster development
• List of errors, where they occurred and possible fix
• General stats about the run of tests just performed

The test ran quickly, but failed. Look over the error. It indicates that Coffee is not defined anywhere. That’s an easy fix.

Add the Coffee class.

# coffee.rb

class Coffee
  def initialize; end
end


# coffee.rb

require 'minitest/autorun'
require_relative 'coffee'

class CoffeeTest < Minitest::Test
  def test_coffee
    assert_instance_of Coffee, Coffee.new
  end
end

And now, run the test again. Same command as before

$ ruby coffee_test.rb


Run options: --seed 5584

# Running:

.

Finished in 0.000671s, 1490.3130 runs/s, 1490.3130 assertions/s.

1 runs, 1 assertions, 0 failures, 0 errors, 0 skips

The test now passes.

That’s all there is to testing using Minitest.

Add another test case

Add a test that will check an instance of Coffee, will return a list of flavours.

The assertion to use here will be assert_includes. Which takes a collection and an obj, validating that the obj is in the collection.

# coffee.rb

class Coffee
  attr_reader :flavours

  def initialize
    @flavours = %w[Chocolate Honey Toffee]
  end
end


# coffee_test.rb

require 'minitest/autorun'
require_relative 'coffee'

class CoffeeTest < Minitest::Test
  def test_coffee
    assert_instance_of Coffee, Coffee.new
  end

  def test_coffee_flavour
    possible_flavour = 'honey'
    tasted_flavours = Coffee.new.flavours.map(&:downcase)
    assert_includes tasted_flavours, possible_flavour
  end
end

How to write tests using Spec

Minitest comes with a Spec DSL. Which, instead of writing methods, they’re written plain english. If you’re familiar with RSpec, you’ll feel at home with Minitest/Spec.

Write your first spec

Continuing with our previous coffee example, we’ll do the following:

• Update each test test_description to use describe
• update each test to use itstatements
• Make use of let statements

Update test descriptions

We’ll start by moving our class Coffee above the tests.

Make sure your test follows the class you’re resting. This wasn’t a requirement before because we were creating a new class. In this case, we’re referencing an existing class.

Next, we’ll replace our class CoffeeTest with a describe block.

# coffee.rb

class Coffee
  ...
end


# coffee_test.rb

require 'minitest/autorun'

describe Coffee do
  ...
end

The big change here is

class CoffeeTest < Minitest::Test

becomes

describe Coffee do

Everything else stays the same.

You can nest describe blocks which can help with organizing larger test files into logical pieces.

We’re now ready to continue upgrading our tests to specs.

Replace test methods with it statements

Next, we’ll replace test_ methods with it statements. This will make our tests read more like plain language, and less like a Ruby method.

# coffee.rb

class Coffee
  ...
end


# coffee_test.rb

require 'minitest/autorun'

describe Coffee do
  it 'gives us a new Coffee' do
    ...
  end

  it 'has a honey flavour' do
    ...
  end
end

it statements define an expectation with a plain english name. If no name is passed in, it will default to anonymous. For readability, it’s recommended to add a name to all it statements.

Under the hood, an it statement gets converted into a test_the_thing method like we had previously.

Replace variable definitions with let statements

let allows you to write a concise accessor that memoizes its contents after the first call to it.

For our existing tests, we don’t have anything that needs it. We’re going to make our possible flavours more complicated than just a string so we can try using let

# coffee.rb

class Coffee
  attr_reader :flavours

  def initialize
    @flavours = %w[Chocolate Honey Toffee]
  end
end


# coffee_test.rb

require 'minitest/autorun'
require_relative 'coffee'

describe Coffee do
  it 'gives us a new Coffee' do
    assert_instance_of Coffee, Coffee.new
  end

  let(:possible_flavour) { %w[chocolate honey toffee].sample }

  it 'has a honey flavour' do
    tasted_flavours = Coffee.new.flavours.map(&:downcase)
    assert_includes tasted_flavours, possible_flavour
  end
end

You’ll notice that we defined the let statement outside of the it block. This is for reuse in other tests. And because it’s memoized, every other test that calls it will use the same value that was returned the first time it was called.

Another common pattern is to put shared let statements at the top or at least above a series of tests that depend on them.

You can only access a let value from inside an it statement. If you try to access it from outside a it statement, the test suite will fail.

Replace class calls with subject.

Subject is a lazy man’s generator. It will return the block you specify as subject.

# coffee.rb

class Coffee
  attr_reader :flavours

  def initialize
    @flavours = %w[Chocolate Honey Toffee]
  end
end


# coffee_test.rb

require 'minitest/autorun'
require_relative 'coffee'

describe Coffee do
  let(:possible_flavour) { %w[chocolate honey toffee].sample }

  subject { Coffee.new }

  it 'gives us a new Coffee' do
    assert_instance_of Coffee, subject
  end

  it 'has a honey flavour' do
    assert_includes subject.flavours.map(&:downcase), possible_flavour
  end
end

You’ll notice that we’ve placed subject above both of our test cases, it’s also not overly complex. But allows us to not write Coffee.new all over the place.

Rake Task

Don’t know what rake is? Rake is a task runner for Ruby. This post won’t show you how to use or write Rake Tasks, but how what a simple task looks like for testing. If you want to learn more about Rake, I recommend Stuart Ellis’s - Using Rake to Automate Tasks.

Add Rake to your Gemfile

gem install rake

Add the following to rakefile.rb.

# rakefile.rb

require 'rake/testtask'

desc 'Run test suite'
Rake::TestTask.new() do |t|
  t.pattern = './**/*_test.rb'
end

This will find all files ending with _test.rb, and then call each using Rake::TestTask.

rake test

Which will output results that looks exactly it you called the test directly. The difference is, the rake task groups all the tests to gether.

Run options: --seed 52969

# Running:

..

Finished in 0.000582s, 3435.1925 runs/s, 5152.7888 assertions/s.

2 runs, 3 assertions, 0 failures, 0 errors, 0 skips

Finish the test suite

You should now have enough information to start implenting coverage on any of your Ruby programs.

In Deeper with Minitest

Now we’ll take a look into more of the features available in Minitest.

Built in assertions

Minitest comes with a ton of assertions built in, which will cover almost every test case you can think of. We’ll go over a few of the more common assertions below.

Minitest assertions always check for a postive result. If you want to test for negative / false results, each assertion has a negative-twin refute. Which take the same params as their positive sibling, but make sure the result is the opposite.

assert(predicate, msg = nil) fails unless predicate returns true

require_relative 'test_helper'

describe Coffee do
  it { assert(Coffee.new.rating == 'Amazing!') }
end

assert_empty(obj, msg = nil) fails unless obj is empty

require_relative 'test_helper'

describe Coffee do
  it { assert_empty Coffee.new.extras }
end

assert_includes(collection, obj, msg = nil) fails unless collection includes obj

require_relative 'test_helper'

describe Coffee do
  subject { Coffee.new.flavours }
  it { assert_includes subject.map(&:downcase), $possible_flavour }
end

assert_match(matcher, obj, message = nil) fails unless matcher =~ obj

require_relative 'test_helper'

describe Coffee do
  matcher = /[a-zA-Z].+\!/
  it { assert_match matcher, Coffee.new.rating }
end

assert_nil(obj, msg = nil) fails unless obj is nil

require_relative 'test_helper'

describe Cup do
  it { assert_nil Cup.new.filling }
end

assert_raise(*exp) fails unless the block raises an exception

require_relative 'test_helper'

describe 'Cup' do
  let(:coffee) { Coffee.new }
  let(:cup) { Cup.new }

  it 'raises when we add another cup' do
    assert_raises RuntimeError do
      cup.add(Cup.new)
    end
  end
end

Minitest comes with a wide range of assertions for testing. You can see all of the available assertions and refutions in the Ruby Docs - Assertions

Using Seeds to replay tests

If you’ve watched the output as you’ve run throug Minispec, you’ll probably have noticed the following

Run options: --seed 59802

A seed is a representation of the order of tests, as well as the randomized data inside your tests. Keeping track of seeds allows you, or someone else to rerun the test exactly as a previous time. This helps for debugging failures.

To specify seeds you need to pass in the value when running your test suite.

If you run tests using the Ruby CLI

ruby test -s 59802

If you run tests using a Rake Task

rake test SEED=59802

Setting up and tearing down tests

As you start building out tests for your code. You’ll notice patterns of setup, start happening around your tests.

Minitest has you covered for wrapping each test with setup code, using setup and teardown

require_relative 'test_helper'

class CoffeeTest < Minitest::Test
  def setup
    @cup = Cup.new
    @cup.add(Coffee.new)
  end

  def teardown
    @cup.drink
  end

  def test_cup_is_not_empty
    refute @cup.empty?
  end
end

If you’re using specs, you can also use before and after

require_relative 'test_helper'

describe 'Coffee' do
  let(:cup) { Cup.new }

  before do
    cup.add(Coffee.new)
  end

  after { cup.drink }

  it 'checks the cup has coffee in it' do
    refute cup.empty?
  end
end

setup, teardown, before, and after will be run around each test. This can add significant delays to your tests if you’re putting complex setup logic in the setup code. You can add the gem minitest-hooks to get access to before_all, after_all which runs around all tests in the test file. This is more commonly used when testing many parts of an instance that has a decent amount of setup

Skipping tests

There comes a time when you are testing your code, and you realize you need to test something else as well. You can remember what you want to write, you can leave yourself a comment, or you can write a skip statement.

Using skip is the same as using any of the assert, refute methods.

require_relative 'test_helper'

describe 'Coffee' do
  it 'checks the coffee rating' do
    assert(Coffee.new.rating == 'Amazing!')
  end

  it 'makes sure the taste is not bitter' do
    skip 'coffee.taste should not be bitter'
  end
end

Skip statements have an advantage over leaving a comment. That is the test reporter will show a list of all the skipped tests, so if you forget one, there is a small indicator to let you know you still have some work to do.

But beware if you use setup/teardown, or before/after to do complex setups, these will still be run around the skipped test. This can increase your test time.

Stubbing

Like other testing libraries, stubs temporarily replace a method and return the specified result. The original method is then swapped back in when the stub block is finished.

Stubs are great for temporarily replacing methods that may take too long to run in your test suite, or are incapable of running in the environment. This does not mean that method should never be tested. Just that the method might be better tested elsewhere.

class Cup
  ...
  def slow_drink
    sleep 10
    puts '*slurp*'
    sleep 10
    puts '*slurp*'
    sleep 10
    @filling = nil
  end
  ...
end

require_relative 'test_helper'

describe 'Cup' do
  let(:coffee) { Coffee.new }
  let(:cup) { Cup.new }

  before { cup.add coffee }

  it 'drinks a coffee slowly' do
    cup.stub :slow_drink, cup.filling = nil do
      cup.slow_drink
      assert_nil cup.filling
    end
  end
end

Unlike a Mock the method needs to exist prior to stubbing. You can’t use a stub on a method that does not exist.

If you must test a method that doesn’t exist, you can use a singleton method to create a new non-existant method

Benchmarking Minitest

Minitest comes with benchmarking baked in. This is handy for testing out performance for the code you write.

There are 2 posts from Chris Kottom that describe Benchmarking with Minitest from a high level, as well as practical examples. I recommend reading them a few times.

Minitest::Benchmark: An Introduction

Minitest::Benchmark: A Practical Example

Minitest & Rails

Rails by default uses Minitest to run your tests. In fact, Rails itself is tested using minitest.

The default Rails test_helper.rb is configured to run test style tests, it looks like the following.

# test_helper.rb

ENV["RAILS_ENV"] = "test"
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'

class ActiveSupport::TestCase
  # Setup all fixtures in test/fixtures/*.(yml|csv) for all tests in alphabetical order.
  #
  # Note: You'll currently still have to declare fixtures explicitly in integration tests
  # -- they do not yet inherit this setting
  fixtures :all

  # Add more helper methods to be used by all tests here...
end

We can add support for Minitest::Spec DSL pretty easily.

# test_helper.rb

ENV["RAILS_ENV"] = "test"
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
require 'minitest/autorub'

class ActiveSupport::TestCase
  # Setup all fixtures in test/fixtures/*.(yml|csv) for all tests in alphabetical order.
  #
  # Note: You'll currently still have to declare fixtures explicitly in integration tests
  # -- they do not yet inherit this setting
  fixtures :all

  # Add more helper methods to be used by all tests here...
  extend MiniTest::Spec::DSL

  register_spec_type(self) do |desc|
    desc < ActiveRecord::Base if desc.is_a?(Class)
  end
end

With spec support inside test_helper.rb you can now write test, specs, and any combination of features to test your Rails code. MAGIC

References

This post only covers a small portion of what Minitest can do. That is not including what the numerous extensions bring to the table.

The following links should help you on your way to mastering Minitest.

• SeattleRB - Minitest
• Ruby Docs - Minitest