[Ruby] How to dynamically write repeated test cases using test/unit(Test::Unit)

4 minute read

Code from theory

Since Ruby can generate code dynamically even Class and Method, it can be written as follows.

ARR = [true, false, true]
Class.new(Test::Unit::TestCase) do
  ARR.each_with_index do |e, idx|
    define_method "test_#{idx}" do
      assert e
    end
  do
end

This is synonymous with writing:

ARR = [true, false, true]
class AnyTestCase <Test::Unit::TestCase
  def test_0
    assert ARR[0]
  end

  def test_1
    assert ARR[1]
  end

  def test_2
    assert ARR[2]
  end
end

Test code is DAMP rather than DRY

It’s often heard that test code should focus on DAMP (Descriptive and Meaningful Phrases) rather than DRY (Don’t Repeat Yourself).

I basically agree with you on this matter. The test code should be a “specification”, and in that sense, the same expression appears repeatedly in “specification written in natural language”, which is unavoidable as a result of prioritizing intelligibility. However, it goes without saying that balance is important in everything.

Well, what is worrisome here is the “verification process for large amounts of data”. For example

  • Verification of consumption tax calculation logic. The number of product masters is 100,000 (^^;;
  • Moving your blog. 1000 entries (one q`)

If you implement it honestly

Let’s target moving blogs. The following is a sneak peek at the blog you’re moving to that there is more than one byte of content inside the <article> tag.

require'test/unit'
require'httpclient'
require'nokogiri'
  
class WebTestCase <Test::Unit::TestCase
  def setup
    @c = HTTPClient.new
  end

  def test_0
    doc = Nokogiri::HTML(@c.get("https://example.com/entry/one").body)
    assert_compare 1, "<", doc.css('article').text.length
  end

  def test_1
    doc = Nokogiri::HTML(@c.get("https://example.com/entry/two").body)
    assert_compare 1, "<", doc.css('article').text.length
  end

  def test_2
    doc = Nokogiri::HTML(@c.get("https://example.com/entry/three").body)
    assert_compare 1, "<", doc.css('article').text.length
  end
end

It would be subtle if I said that I would write up to def test_1000 later.

Implementation that iterator repeats

Since the verification condition (assert_compare) does not change, it is possible to think of using the URL list as an iterator (array) to test it repeatedly, but this is not recommended for the reasons described below.

require'test/unit'
require'httpclient'
require'nokogiri'

URLS = %w(
  https://example.com/entry/one
  https://example.com/entry/two
  https://example.com/entry/three
)

class WebTestCase <Test::Unit::TestCase
  def setup
    @c = HTTPClient.new
  end

  def test_content_length
    URLS.each do |url|
      doc = Nokogiri::HTML(@c.get(url).body)
      assert_compare 1, "<", doc.css('article').text.length
    end
  end
end

%w notation is convenient. Well, it looks good. However, the problem actually occurs when assert fails. The following is the execution result, but I do not know at which URL it failed.

$ bundle exec ruby test_using_array.rb
Started
F
================================================== ================================================== ================================================== ===================================
test_using_array.rb:25:in `test_content_length'
test_using_array.rb:25:in `each'
     24: def test_content_length
     25: URLS.each do |url|
     26: doc = Nokogiri::HTML(@c.get(url).body)
  => 27: assert_compare 1, "<", doc.css('article').text.length
     28: end
     29: end
     30: end
test_using_array.rb:27:in `block in test_content_length'
Failure: test_content_length(WebTestCase):
  <1> <<0> should be true
  <1> was expected to be less than
  <0>.
================================================== ================================================== ================================================== ===================================

Finished in 0.0074571 seconds.
- ------------------------------------------------- -------------------------------------------------- -------------------------------------------------- -----------------------------------
1 tests, 3 assertions, 1 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
0% passed
- ------------------------------------------------- -------------------------------------------------- -------------------------------------------------- -----------------------------------

This is because the unit of test is each method (def test_content_length in the above example).

If you want to generate code, you should let Ruby create it dynamically.

For each iterator (array), we want to create a method for each test and test it in it. After all, is it foolish to copy and paste it? Is it only possible to generate code with Hidemaru’s macro? When you come to this thought, remember that dynamic code generation, which is Ruby’s black magic (metaprogramming).

As I mentioned at the beginning, Ruby can generate code dynamically. Class and Method are no exception. With it, you can dynamically create test methods for each of the contents of the iterator.

URLS = %w(
  https://example.com/entry/one
  https://example.com/entry/two
  https://example.com/entry/three
)

Class.new(Test::Unit::TestCase) do
  def setup
    @c = HTTPClient.new
  end

  URLS.each_with_index do |url, idx|
    define_method "test_#{idx}" do
      doc = Nokogiri::HTML(@c.get(url).body)
      assert_compare 1, "<", doc.css('article').text.length
    end
  end
end

This code is equivalent to the following and is the same as the naive implementation.

URLS = %w(
  https://example.com/entry/one
  https://example.com/entry/two
  https://example.com/entry/three
)

class AnyTestCase <Test::Unit::TestCase
  def setup@c = HTTPClient.new
  end

  def test_0
    doc = Nokogiri::HTML(@c.get(URLS[0]).body)
    assert_compare 1, "<", doc.css('article').text.length
  end

  def test_1
    doc = Nokogiri::HTML(@c.get(URLS[1]).body)
    assert_compare 1, "<", doc.css('article').text.length
  end

  def test_2
    doc = Nokogiri::HTML(@c.get(URLS[2]).body)
    assert_compare 1, "<", doc.css('article').text.length
  end
end

以下の実行例は、失敗した場所が Failure: test_2() と明示されています。

$ bundle exec ruby test_using_black_magic.rb
Loaded suite test_using_black_magic
Started
..F
=========================================================================================================================================================================================
test_using_black_magic.rb:27:in `block (3 levels) in <main>'
Failure: test_2():
  <1> < <0> should be true
  <1> was expected to be less than
  <0>.
=========================================================================================================================================================================================

Finished in 0.007288 seconds.
- ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
3 tests, 3 assertions, 1 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
66.6667% passed
- ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

#each_with_index は 0 オリジン(はじまり)です。 test_2 で失敗したという事は ARR[2] の値の時に失敗したという事になるので、その値を検証していけばよいわけです。

--name での指定にも対応できる

イテレーターに格納された値の順番が保証されていれば --name での指定にも対応してくれます。例えば ARR[1] の値に対してのみテストを行いたい場合は、以下のように指定します。

$ bundle exec ruby test_multiple.rb --name test_1

実装のポイント

Class.new による動的クラス生成と define_method による動的メソッド生成がキーです。 このへんに興味が出たら “Ruby 黒魔術” とかで検索してみてください。

やりすぎに注意

assertion 条件が画一的な時には有効です。 逆に、入力された値によって適用する assertion が異なる(= 条件分岐が発生する)ようであれば、愚直に実装した方が DAMP となります。 ※そもそもテストケース内で条件分岐が発生している時点で、テストケース自体を見直すべきでしょう。

あとがき

この分野は素人なので、これが合ってるのかわからん。

EoT