[Ruby] Digdag changes the setting value for each environment (RubyOnRails)

3 minute read

About #Digdag Getting started Architecture Concepts Workflow definition Scheduling workflow Operators Command reference Language API -Ruby Change the setting value for each environment with Digdag (RubyOnRails) Batch implementation in RubyOnRails environment using Digdag

things to do

I will create a YAML format environment setting file for Digdag in the sample application of the Ruby on Rails tutorial, get the environment variables from the Ruby task, set it in the Digdag environment variable, and check until you can get the Digdag environment variable from another Ruby task ..

The sample application code from the Ruby on Rails tutorial is irrelevant to this explanation, but since it will be used in the next batch implementation, we will download it and add the Digdag Workflow to that project.

Ruby on Rails Tutorial Sample Application Download

https://github.com/yasslab/sample_app ![Screenshot 2020-07-15 23.06.34.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/108475/43a1cd77-9f55-3be3-9956-(5db704cf4e34.png)

workflow generation

$ digdag init workflow
$ cd workflow
$ digdag run workflow.dig

If the following result is displayed, there is no problem in the workflow environment of Digdag ~

result


2020-07-15 22:01:56 +0900 [INFO](0017@[0:default]+workflow+setup): echo>: start 2020-07-15T00:00:00+00:00
start 2020-07-15T00:00:00+00:00
2020-07-15 22:01:57 +0900 [INFO](0017@[0:default]+workflow+disp_current_date): echo>: 2020-07-15 00:00:00 +00:00
2020-07-15 00:00:00 +00:00
2020-07-15 22:01:57 +0900 [INFO](0017@[0:default]+workflow+repeat): for_each>: {order=[first, second, third], animal=[dog, cat] }
2020-07-15 22:01:57 +0900 [INFO](0018@[0:default]+workflow+repeat^sub+for-0=order=0=first&1=animal=1=cat): echo>: first cat
first cat
2020-07-15 22:01:57 +0900 [INFO](0019@[0:default]+workflow+repeat^sub+for-0=order=1=second&1=animal=0=dog): echo>: second dog
second dog
2020-07-15 22:01:58 +0900 [INFO](0020@[0:default]+workflow+repeat^sub+for-0=order=1=second&1=animal=1=cat): echo>: second cat
second cat
2020-07-15 22:01:58 +0900 [INFO](0022@[0:default]+workflow+repeat^sub+for-0=order=2=third&1=animal=1=cat): echo>: third cat
third cat
2020-07-15 22:01:58 +0900 [INFO](0021@[0:default]+workflow+repeat^sub+for-0=order=2=third&1=animal=0=dog): echo>: third dog
third dog
2020-07-15 22:01:58 +0900 [INFO](0017@[0:default]+workflow+repeat^sub+for-0=order=0=first&1=animal=0=dog): echo>: first dog
first dog
2020-07-15 22:01:58 +0900 [INFO](0017@[0:default]+workflow+teardown): echo>: finish 2020-07-15T00:00:00+00:00
finish 2020-07-15T00:00:00+00:00

Create environment variable file in YAML format

$ cd workflow
$ mkdir config
$ vi environment.yml

I set url, user and password for development, staging and production. Reference this environment variable from the task of Digdag.

environment.yml


development:
  url: development.jp
  user: development_user
  password: development_password

staging:
  url: staging.jp
  user: staging_user
  password: staging_password
  
production:
  url: procuction.jp
  user: procuction_user
  password: procuction_password

Create a Ruby class that sets the environment variable of the YAML file to the environment variable of Digdag.

Create a tasks folder under the workflow folder Create environment.rb there. This class will read the environment variables from the YAML file and store them in Digdag’s environment variables.

$ cd workflow
$ mkdir tasks

environment.rb


require'yaml'

class Environment
  def load
    environment = Digdag.env.params['environment']
    environment = case environment
                  when'stg'
                    'staging'
                  when'prd'
                    'production'
                  else
                    'development'
                  end

# Get the environment variable of the corresponding environment
    config = YAML.load_file('config/environment.yml')[environment]
    config.each do |key, value|
  # Save to Digdag environment variable
      Digdag.env.store(key.to_sym => value)
    end
  end
end

Add task to Workflow.dig

Delete the code automatically generated by $ digdag init workflow and add the following code

workflow.dig


timezone: UTC

+load_environment:
  rb>: Environment.load
  require:'tasks/environment'
+print_environment_value:
  sh>: echo url:${url} user:${user} password:${password}
+print_ruby:
  rb>: DigdagEnvironment.show
  require:'tasks/digdag_environment'

■ load_environment task: A task that sets the relevant environment variables to the Digdag environment variables from the environment parameters when Digdag is executed ■ print_environment_value: Task that outputs the Digdag environment variable from the shell task ■ print_ruby: Task to output Digdag environment variables from Ruby class

Let’s run it from the development environment.

$digdag run workflow.dig -p environment=development --rerun

development environment execution result


2020-07-15 22:49:35 +0900 [INFO](0017@[0:default]+workflow+load_environment): rb>: Environment.load
2020-07-15 22:49:35 +0900 [INFO](0017@[0:default]+workflow+print_environment_value): sh>: echo url:development.jp user:development_user password:development_password
url:development.jp user:development_user password:development_password
2020-07-15 22:49:36 +0900 [INFO](0017@[0:default]+workflow+print_ruby): rb>: DigdagEnvironment.showurl: development.jp
user: development_user
password: development_password

次はStaging環境

$digdag run workflow.dig -p environment=stg --rerun

staging環境で実行結果


2020-07-15 22:55:33 +0900 [INFO](0017@[0:default]+workflow+load_environment): rb>: Environment.load
2020-07-15 22:55:33 +0900 [INFO](0017@[0:default]+workflow+print_environment_value): sh>: echo url:staging.jp user:staging_user password:staging_password
url:staging.jp user:staging_user password:staging_password
2020-07-15 22:55:34 +0900 [INFO](0017@[0:default]+workflow+print_ruby): rb>: DigdagEnvironment.show
url: staging.jp
user: staging_user
password: staging_password

最後にProduction環境

$digdag run workflow.dig -p environment=prd --rerun

production環境で実行結果


2020-07-15 22:56:30 +0900 [INFO](0017@[0:default]+workflow+load_environment): rb>: Environment.load
2020-07-15 22:56:30 +0900 [INFO](0017@[0:default]+workflow+print_environment_value): sh>: echo url:procuction.jp user:procuction_user password:procuction_password
url:procuction.jp user:procuction_user password:procuction_password
2020-07-15 22:56:31 +0900 [INFO](0017@[0:default]+workflow+print_ruby): rb>: DigdagEnvironment.show

これで環境変数設定が終わったので次回からはRubyOnRailsのチュートリアルアプリのデータでバッチを作ってみようと思います。