Batch implementation in RubyOnRails environment using Digdag

About Digdag

Getting started Architecture Concepts Workflow definition Scheduling workflow Operators Command reference Language API -Ruby Change the setting value for each environment with Digdag (RubyOnRails) Batch implementation in RubyOnRails environment using Digdag

Preparation

Download sample application for Ruby on Rails tutorial https://github.com/yasslab/sample_app

Run the sample application of the Ruby on Rails tutorial to register users and Posts. I created a test user and registered 4 posts.

This time, we will create a simple batch that outputs the number of posts of the corresponding user with the user name as a parameter. There are two ways to implement and start the batch.

Let's try the following two methods that are often used when writing batch processing in Rails. ① rails runner: Write a batch as a script ② rake task: Write a batch as a build task

Overview

Run with rails runner

I used OptionParser to get the parameters.

Added batch processing script

/lib/scripts/Added batch processing script under

lib/scripts/post_batch.rb

require 'optparse'

module Scripts
  class PostBatch
    def initialize
      @option = {}
      OptionParser.new do |opt|
        opt.on('-n VALUE', 'user name') { |v| @option[:name] = v}
        opt.parse!(ARGV)
      end
    end

    def count
      user = User.find_by(name: @option[:name])
      puts "ID: #{user.id}name:#{user.name}"
      puts "Number of posts:#{Micropost.where(user_id: user.id).count}"
    end
  end
end

Automatically load ruby files under lib

config/application.rb

config.autoload_paths += %W(#{config.root}/lib)

Add rails_runner.dig under workflows and add the following contents

rails_runner.dig

+task:
  sh>: bundle exe rails runner Scripts::PostBatch.new.count -n 'test'

Run

The ID, name, and number of posts of the test user are output.

Execution result

$ digdag run rails_runner.dig --rerun
2020-07-20 19:38:39 +0900 [INFO](0017@[0:default]+rails_runner+task): sh>: bundle exec rails runner Scripts::PostBatch.count 'test'
ID:5 Name: test
Number of posts: 4

Run with rake

rake task generation

python

$ rails g task task_post
Running via Spring preloader in process 4255
      create  lib/tasks/tast_post.rake

/lib/tasks/task_post.A rake file will be generated, so open it and add the following source

#### **`lib/tasks/task_post.rake`**
```rb

namespace :task_post do
  desc "Get the number of user posts"
  task :count, ['name'] => :environment do |task, args|
    user = User.find_by(name: args.name)
    puts "ID: #{user.id}name:#{args.name}"
    puts "Number of posts:#{Micropost.where(user_id: user.id).count}"
  end
end

Workflow added

rake.dig

+task:
  sh>: bundle exec rake task_post:count[test4]

Run

The ID, name, and number of posts of the test user are output.

Execution result

$ digdag run rake.dig --rerun
2020-07-20 20:04:16 +0900 [INFO](0017@[0:default]+rake+task): sh>: bundle exec rake task_post:count[test]
ID:5 Name: test
Number of posts: 4