Change the setting value for each environment with Digdag (RubyOnRails)

About Digdag

Getting started Architecture Concepts Workflow definition Scheduling workflow Operators Command reference Language API -Ruby Change the setting value for each environment with Digdag (RubyOnRails) Batch implementation in RubyOnRails environment using Digdag

things to do

I will create a YAML-format environment setting file for Digdag in the sample application of the Ruby on Rails tutorial, get the environment variable from the Ruby task, set it in the Digdag environment variable, and check until I can get the Digdag environment variable from another Ruby task. ..

The sample application code for the Ruby on Rails tutorial is irrelevant to this discussion, but we'll townload it and add the Digdag Workflow to that project for use in the next batch implementation.

Download sample application for Ruby on Rails tutorial

https://github.com/yasslab/sample_app スクリーンショット 2020-07-15 23.06.34.png

workflow generation

$ digdag init workflow
$ cd workflow
$ digdag run workflow.dig

If the following result is displayed, there is no problem in the Workflow environment of Digdag ~

result


2020-07-15 22:01:56 +0900 [INFO]([email protected][0:default]+workflow+setup): echo>: start 2020-07-15T00:00:00+00:00
start 2020-07-15T00:00:00+00:00
2020-07-15 22:01:57 +0900 [INFO]([email protected][0:default]+workflow+disp_current_date): echo>: 2020-07-15 00:00:00 +00:00
2020-07-15 00:00:00 +00:00
2020-07-15 22:01:57 +0900 [INFO]([email protected][0:default]+workflow+repeat): for_each>: {order=[first, second, third], animal=[dog, cat]}
2020-07-15 22:01:57 +0900 [INFO]([email protected][0:default]+workflow+repeat^sub+for-0=order=0=first&1=animal=1=cat): echo>: first cat
first cat
2020-07-15 22:01:57 +0900 [INFO]([email protected][0:default]+workflow+repeat^sub+for-0=order=1=second&1=animal=0=dog): echo>: second dog
second dog
2020-07-15 22:01:58 +0900 [INFO]([email protected][0:default]+workflow+repeat^sub+for-0=order=1=second&1=animal=1=cat): echo>: second cat
second cat
2020-07-15 22:01:58 +0900 [INFO]([email protected][0:default]+workflow+repeat^sub+for-0=order=2=third&1=animal=1=cat): echo>: third cat
third cat
2020-07-15 22:01:58 +0900 [INFO]([email protected][0:default]+workflow+repeat^sub+for-0=order=2=third&1=animal=0=dog): echo>: third dog
third dog
2020-07-15 22:01:58 +0900 [INFO]([email protected][0:default]+workflow+repeat^sub+for-0=order=0=first&1=animal=0=dog): echo>: first dog
first dog
2020-07-15 22:01:58 +0900 [INFO]([email protected][0:default]+workflow+teardown): echo>: finish 2020-07-15T00:00:00+00:00
finish 2020-07-15T00:00:00+00:00

Creating an environment variable file in YAML format

$ cd workflow
$ mkdir config
$ vi environment.yml

I set url, user, password for each development, staging and production. Browse this environment variable from the Digdag task.

environment.yml


development:
  url: development.jp
  user: development_user
  password: development_password

staging:
  url: staging.jp
  user: staging_user
  password: staging_password
  
production:
  url: procuction.jp
  user: procuction_user
  password: procuction_password

Create a Ruby class that sets the environment variables in the YAML file to the environment variables in Digdag.

workflowUnder the foldertasksCreate a folder Create environment.rb there. This class reads environment variables from a YAML file and saves them in Digdag's environment variables.

$ cd workflow
$ mkdir tasks

environment.rb


require 'yaml'

class Environment
  def load
    environment = Digdag.env.params['environment']
    environment = case environment
                  when 'stg'
                    'staging'
                  when 'prd'
                    'production'
                  else
                    'development'
                  end

    #Get the environment variables of the corresponding environment
    config = YAML.load_file('config/environment.yml')[environment]
    config.each do |key, value|
      #Save to Digdag environment variable
      Digdag.env.store(key.to_sym => value)
    end
  end
end

Add task to Workflow.dig

$ digdag init workflowDelete the code automatically generated by and add the following code

workflow.dig


timezone: UTC

+load_environment:
  rb>: Environment.load
  require: 'tasks/environment'
+print_environment_value:
  sh>: echo url:${url} user:${user} password:${password}
+print_ruby:
  rb>: DigdagEnvironment.show
  require: 'tasks/digdag_environment'

■ load_environment task: A task to set the corresponding environment variable to the Digdag environment variable from the environment parameters when Digdag is executed. ■ print_environment_value: A task that outputs Digdag environment variables from a shell task ■ print_ruby: Task to output Digdag environment variable from Ruby class

Try running it from the development environment.

$digdag run workflow.dig -p environment=development --rerun

Execution result in development environment


2020-07-15 22:49:35 +0900 [INFO]([email protected][0:default]+workflow+load_environment): rb>: Environment.load
2020-07-15 22:49:35 +0900 [INFO]([email protected][0:default]+workflow+print_environment_value): sh>: echo url:development.jp user:development_user password:development_password
url:development.jp user:development_user password:development_password
2020-07-15 22:49:36 +0900 [INFO]([email protected][0:default]+workflow+print_ruby): rb>: DigdagEnvironment.show
url: development.jp
user: development_user
password: development_password

Next is the Staging environment

$digdag run workflow.dig -p environment=stg --rerun

Execution result in staging environment


2020-07-15 22:55:33 +0900 [INFO]([email protected][0:default]+workflow+load_environment): rb>: Environment.load
2020-07-15 22:55:33 +0900 [INFO]([email protected][0:default]+workflow+print_environment_value): sh>: echo url:staging.jp user:staging_user password:staging_password
url:staging.jp user:staging_user password:staging_password
2020-07-15 22:55:34 +0900 [INFO]([email protected][0:default]+workflow+print_ruby): rb>: DigdagEnvironment.show
url: staging.jp
user: staging_user
password: staging_password

Finally the Production environment

$digdag run workflow.dig -p environment=prd --rerun

Execution result in production environment


2020-07-15 22:56:30 +0900 [INFO]([email protected][0:default]+workflow+load_environment): rb>: Environment.load
2020-07-15 22:56:30 +0900 [INFO]([email protected][0:default]+workflow+print_environment_value): sh>: echo url:procuction.jp user:procuction_user password:procuction_password
url:procuction.jp user:procuction_user password:procuction_password
2020-07-15 22:56:31 +0900 [INFO]([email protected][0:default]+workflow+print_ruby): rb>: DigdagEnvironment.show

Now that the environment variables have been set, I will try to create a batch from the data of the tutorial app of RubyOnRails from the next time.

Recommended Posts

Change the setting value for each environment with Digdag (RubyOnRails)
Change the injection target for each environment with Spring Boot 2
Specify the timeout for each path with Rack :: Timeout
Prepare the environment for java11 and javaFx with Ubuntu 18.4
Change the port with SpringBoot
[Android] Change the app name and app icon for each Flavor
[Rails] How to change the page title of the browser for each page
Java_HOME setting for paths with spaces
Change the processing when the button on RecyclerView is pressed for each list
Problems I was addicted to when building the digdag environment with docker
Change the default timezone for the rails app
Prepare the format environment with "Rails" (VScode)
Prepare the security check environment for Rails 6
Change the layout when rotating with onConfigurationChanged