[RUBY] Implementation example of centrally managing inventory and variables by linking Ansible and serverspec (supports multiple servers)

Introduction

For the Linux server group set in Ansible, I tried to modify (?) Ansible and serverspec in order to solve the following problems that I had when testing with serverspec.

Please refer to it as an introductory / beginner's edition when using Ansible and serverspec in cooperation.

goal

Implement so that you can:

Implementation example

environment

Ansible Control Node

Ansible Managed Node

Directory structure

$ tree -aF /autotools
/autotools
|-- .ssh/
|   `-- aws_key.pem    #Managed Node SSH private key
|-- ansible/
|   |-- ansible.cfg
|   |-- group_vars/    #Variable directory for groups
|   |-- host_vars/     #Variable directory for host
|   |-- inventory/     #Inventory placement directory for Ansible
|   `-- centos.yml       # Playbook
`-- serverspec/
    |-- .rspec
    |-- Rakefile
    |-- spec/
    |   |-- base/      #Test coat placement directory for base role
    |   |   `-- sample_spec.rb #Test code
    |   `-- spec_helper.rb
    `-- spec_hosts/    #Variable placement directory for serverspec

Ansible

First, let's decide project_name with an English character string for the purpose of managing this server group collectively. Here, as an example, use project_name as ʻanken`.

ansible.cfg Here, specify ʻansible.cfg to be used in the environment variable ʻANSIBLE_CONFIG. I'm using the same hostname in my code development, so I've listed the ssh arguments here.

$ export ANSIBLE_CONFIG=/autotools/ansible/ansible.cfg

/autotools/ansible/ansible.cfg


[defaults]

[ssh_connection]
ssh_args = -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no

[privilege_escalation]
become = true

SSH private key

Place the SSH private key used to SSH into the Ansible Managed Node in /autotools/.ssh. The key path is listed in the inventory file.

Inventory file

Place the inventory file ʻanken.iniunder/ autotools / ansible / inventory /`.

Basically, it is OK if you write according to Ansible rules, but since it is used in the mechanism for linking with serverspec, please set the following three indispensable.

/autotools/ansible/inventory/anken.ini


[anken]
prod_foobar1 ansible_host=xx.xx.xx.xx
dev_foobar1 ansible_host=yy.yy.yy.yy

[anken:vars]
ansible_user=centos
ansible_ssh_private_key_file=~/.ssh/aws_key.pem

[all:vars]
project_name=anken

Ansible is executed for the IP or name of ʻansible_host`.

ʻInventory_hostname (where prod_foobar1 dev_foobar2` is specified in the example) does not have to match the actual hostname of the node.

Playbook

The playbook example used in this implementation example is as follows. name: Configure for serverspec at localhost outputs the variable file used by serverspec.

centos.yml


---
- name: Playbook for centos7 managed node
  hosts: all
  gather_facts: true

  tasks:
    - name: Create group
      group:
        name: "{{ item.name }}"
        gid: "{{ item.gid }}"
      loop: "{{ group }}"
      tags: group

    - name: Create User
      user:
        name: "{{ item.name }}"
        uid: "{{ item.uid }}"
        group: "{{ item.group }}"
        groups: "{{ item.groups }}"
        home: "{{ item.home }}"
        shell: "{{ item.shell }}"
      loop: "{{ user }}"
      tags: user

    - name: System service
      systemd:
        name: "{{ item.name }}"
        enabled: "{{ item.enabled }}"
        state: "{{ item.state }}"
      loop: "{{ service }}"
      tags: service

- name: Configure for serverspec at localhost
  hosts: localhost
  connection: local
  gather_facts: false
  tasks:

    - name: Dump hostvars for serverspec
      copy:
        content: "{{ hostvars | to_nice_yaml }}"
        dest: "../serverspec/spec_hosts/{{ project_name }}.yml"
      tags: serverspec

Variable file placement

Variables common to projects are placed in /autotools/ansible/group_vars/#{project_name}.yml. If you want to specify a different variable only for a specific inventory_hostname, place it in /autotools/ansible/host_vars/#{inventory_name}.yml.

In the variable file, specify the role to use when testing with serverspec with serverspec_role.

/autotools/ansible/group_vars/anken.yml


serverspec_role:
  - base
group:
  - name: unyo
    gid: 1101
  - name: infra
    gid: 1102
  - name: app
    gid: 1103
user:
  - name: user1
    uid: 2001
    group: customer
    groups: [ unyo, infra]
    home: /home/user1
    shell: /bin/bash
  - name: user2
    uid: 2002
    group: customer
    groups: [ app ]
    home: /home/user2
    shell: /bin/bash
  - name: user3
    uid: 2003
    group: customer
    groups: [ app, infra ]
    home: /home/user3
    shell: /bin/bash
service:
  - name: chronyd.service
    enabled: false
    state: stopped
  - name: rsyncd.service
    enabled: true
    state: started

Here, I will limit it to the prod_foobar node and overwrite some variables.

/autotools/ansible/group_vars/prod_foobar.yml


group:
  - name: unyo
    gid: 2101
  - name: infra
    gid: 2102
  - name: app
    gid: 2103

Directory / file structure

After arranging the necessary files, it should look like this.

$ tree /autotools/ansible -aF
/autotools/ansible
|-- ansible.cfg
|-- centos.yml
|-- group_vars/
|   `-- anken.yml
|-- host_vars/
|   `-- prod_foobar1
`-- inventory/
    `-- anken.ini

Run

Run the centos.yml playbook with an inventory file as shown below.

$ cd /autotools/ansible
$ ansible -i ./inventory/anken centos.yml

serverspec

Directory / file structure

After running Ansible, the directory / file structure on the serverspec side should look like this.

# tree /autotools/serverspec -aF
/autotools/serverspec
|-- .rspec
|-- Rakefile
|-- spec/
|   |-- base/
|   |   `-- sample_spec.rb
|   `-- spec_helper.rb
`-- spec_hosts/
    `-- anken.yml        #Variable file generated by Ansible

Execution command

The order is different, but the execution command of serverspec is as follows. This is an image of passing the variable file name (generated by Ansible) used in serverspec to the rake command as an argument.

$ rake spec anken -T
rake spec               # Run spec to all hosts
rake spec:dev_foobar1   # Run spec to dev_foobar1
rake spec:prod_foobar1  # Run spec to prod_foobar1
$ rake spec anken

Rakefile

There are some changes from the standard Rakefile created by serverspec-init.

/autotools/serverspec/Rakefile


require 'rake'
require 'rspec/core/rake_task'
require 'yaml'

#Read variable file
project_name = ARGV[1]
hosts = YAML.load_file("./spec_hosts/#{project_name}.yml")

desc "Run spec to all hosts"
task :spec => 'spec:all'

namespace :spec do
  task :all => hosts.keys.map {|key| 'spec:' + key }

  hosts.keys.each do |key|
    desc "Run spec to #{key}"
    RSpec::Core::RakeTask.new(key.to_sym) do |t|
      ENV['INVENTORY_HOST'] = key
      ENV['PROJECT_NAME'] = project_name
      # serverspec_Under a directory with the same name as role*_spec.Read rb file
      t.pattern = 'spec/{' + hosts[key]['serverspec_role'].join(',') + '}/*_spec.rb'
      t.fail_on_error = false
    end
  end
end

#Forged the argument of the rake command as an empty task
ARGV.slice(1,ARGV.size).each{|v| task v.to_sym do; end}

I used it as a reference below. Thank you very much.

Reference: Write ordinary argument-like processing in Rake task https://qiita.com/nao58/items/aa50514d97f05eb8d128

Reference: Official How to use host specific properties https://serverspec.org/advanced_tips.html

spec_helper.rb

This also changes some functions from the initial state of spec_helper.rb.

/autotools/serverspec/spec/spec_helper.rb


require 'serverspec'
require 'pathname'
require 'net/ssh'
require 'yaml'

#Read variable yml file
key = ENV['INVENTORY_HOST']
project_name = ENV['PROJECT_NAME']
properties = YAML.load_file("./spec_hosts/#{project_name}.yml")
set_property properties["#{key}"]

set :backend, :ssh
set :path, '/sbin:/usr/sbin:$PATH'

#ssh execution part
RSpec.configure do |c|
  c.before :all do
    #Extract the host, user, password or key used in Ansible from the read variable file
    set :host, property['ansible_host']
    options = Net::SSH::Config.for(c.host)
    options[:user] = property['ansible_user']
    if property['ansible_password']
      options[:password] = property['ansible_password']
    else
      options[:keys] = [ property['ansible_ssh_private_key_file'] ]
    end
    options[:user_known_hosts_file] = '/dev/null'
    set :ssh_options, options
  end
end

This spec_helper.rb does not support WinRM because it is set: backend,: ssh. However, since you can write anything in Ruby, it shouldn't be difficult to support Windows.

Test code

This is a sample. As written in spec_helper.rb,property ['xxx']can be used to retrieve a variable from a variable file and reuse it.

/autotools/serverspec/spec/base/


# frozen_string_literal: true

require 'spec_helper'

puts "\nRun serverspec to #{property['inventory_hostname']}"

property['group'].each do |attr|
  describe group(attr['name']) do
    it { should exist }
    it { should have_gid attr['gid'] }
  end
end

property['user'].each do |attr|
  describe user(attr['name']) do
    it { should exist }
    it { should have_uid attr['uid'] }
    it { should belong_to_group attr['group'] }
  end
end

property['service'].each do |attr|
  describe service(attr['name']) do
    attr['enabled']            ? it { should be_enabled } : it { should_not be_enabled }
    attr['state'] == 'started' ? it { should be_running } : it { should_not be_running }
  end
end

Run

Again, execute the rake spec command with the variable file name generated by Ansible as an argument as shown below. It is also possible to execute tests for each unit.

$ rake spec anken
$
$ rake spec anken -T    #Command to display task list
rake spec               # Run spec to all hosts
rake spec:dev_foobar1   # Run spec to dev_foobar1
rake spec:prod_foobar1  # Run spec to prod_foobar1
$
$ rake spec:dev_foobar1 anken

Summary

With Ansible and serverspec, we were able to unify variable files and event refiles, which tend to be double-managed. In addition, I was able to manage and execute the test code on a role-by-role basis by referring to the method described in the serverspec formula.

Since serverspec is a pretty Ruby-colored tool, it's hard for people who don't usually touch Ruby, but once you get used to it, it's good that various processes are easy to write.

Sample code

It is published below. https://github.com/kentarok/autotools

Recommended Posts

Implementation example of centrally managing inventory and variables by linking Ansible and serverspec (supports multiple servers)
Implement a model with state and behavior (3) --Example of implementation by decorator
Example of using class variables and class methods