Try setting NETCONF (ncclient) from software to the router

This is the 20th day post of NetOpsCoding AdventCalender. I'm sorry I was late.

Overview

I wrote this article two weeks ago. Try setting from software to router with SSH (Exscript)

One of the challenges for auto-configuring routers is that the router CLI parsing process is extremely difficult. Since the output result of the status check command in the router CLI is created on the assumption that it is visually checked by a person, it is subtle when trying to automatically extract information using the SSH or Telnet package provided in the programming language. It is very difficult because it is necessary to flexibly handle line breaks, spaces, special characters, error display, etc.

This story is also introduced in @ yuyarin's blog, so please refer to it. What is required of network equipment for operation automation

In order to get rid of the trouble of CLI analysis processing / exception handling by SSH and Telnet package, this time I would like to make software using NETCONF, which is attracting attention in automation. To be honest, the implementation status varies depending on the manufacturer, so I'm not sure if it can be used immediately at work, but this time I will try it for studying as well.

What is NETCONF

NETCONF is a standardized protocol for setting network devices and acquiring information. NETCONF has emerged as an alternative to SNMP, but it is expected to be used in network devices from multiple vendors, and has attracted attention in combination with recent automation and the SDN boom.

All config settings and status output in NETCONF are defined in XML-RPC format, and the parsing process as described above can be done with the minimum necessary.

When I looked it up about a year ago, it seemed that NETCONF was almost implemented in the latest OSs of major manufacturers. However, the implementation status and supported functions differ greatly depending on the OS version, so it is recommended to check whether NETCONF is supported by your version.

Manufactor OS Supported API
Cisco IOS or IOS XE OnePK, NETCONF
IOS XR OnePK, NETCONF
NX OS OnePK, NX-API, NETCONF
Juniper JUNOS JUNOS XML Protocol, REST API, NETCONF
Brocade NetIron REST API, NETCONF
Network OS REST API, NETCONF
Arista EOS eAPI (REST API)

NETCONF reference material

To get an overview of NETCONF, @codeout has a detailed explanation, so you can refer to this article.

NETCONF I don't know Let's try NETCONF

The background and standardization trends of the appearance of NETCONF and YANG models are introduced in the following presentation materials. JANOG36 NETCONF/YANG

If you want to know more details, read the RFC. RFC6241 Network Configuration Protocol (NETCONF) RFC6020 YANG - A Data Modeling Language for the Network Configuration Protocol (NETCONF)

Try using NETCONF

Now, let's use NETCONF to set up the router. Requests and responses are exchanged between the NETCONF client and router using XML-RPC.

The implementation flow is as follows.

  1. Examine the XML-RPC schema (rules structured by tags) defined in the target router OS
  2. Create the content you want to configure according to the XML-RPC format and send it to the router using the NETCONF client.
  3. Check the response message from the router
  4. A person confirms the settings, makes a decision, and executes a commit.

Implementation environment

--Router OS - JUNOS --Programming language

Advance preparation

Install ncclient on the server where you write the Python program.

sudo pip install ncclient

Enter the settings to allow netconf in advance in the JUNOS router.

set system services netconf ssh

Investigate the XML-RPC format of the router

The XML-RPC format of the router is published on each vendor's support page.

NETCONF XML Management Protocol Developer Guide

In JUNOS, it is very convenient to add "| display xml rpc" to the end of the command in the router CLI because the config contents and show command can be output in RPC request format. Use the part enclosed by \ ... \ </ rpc> in NETCONF.

user1@router> show interfaces xe-0/0/0 terse | display xml rpc

<rpc-reply xmlns:junos="http://xml.juniper.net/junos/xxxxx/junos">
    <rpc>
        <get-interface-information>
                <terse/>
                <interface-name>xe-0/0/0</interface-name>
        </get-interface-information>
    </rpc>
    <cli>
        <banner></banner>
    </cli>
</rpc-reply>

Acquisition of status information by show command

By using the command function in ncclient, you can use the show command in the router CLI as it is and get the information without using the XML-RPC schema. Here, execute the command to get the state of interface xe-0 / 0/0.

show_junos.py


#! /usr/bin/env python
# -*- coding: utf-8 -*-

from ncclient import manager

username = 'user1'
password = 'pass1'
ipv4 = '192.168.0.1'
port = 22

connection = manager.connect(host = ipv4, port = port, username = username, password = password, timeout = 20, device_params={'name':'junos'}, hostkey_verify=False )

print '1. run show command'
print '='*40

print connection.command('show interfaces xe-0/0/0  terse')

Here is the execution result. Although the space is slightly off, you can see that you can check the interface status with \ and \ .

Execution result


$ python show_junos.py
1. run show command
========================================
<rpc-reply message-id="urn:uuid:xxxxxx-xxxxxx">
  <interface-information style="terse">
    <physical-interface>
      <name>
xe-0/0/0
</name>
      <admin-status>
down
</admin-status>
      <oper-status>
down
</oper-status>
    </physical-interface>
  </interface-information>
</rpc-reply>

If the specified interface does not exist, an error will be output with the tag \ .

Execution result


[t-tsuchiya@dev set_router_netconf]$ python show_junos.py
1. run show command
========================================
<rpc-reply message-id="urn:uuid:xxxxxx-xxxxxx">
  <interface-information style="terse">
    <rpc-error>
      <error-type>protocol</error-type>
      <error-tag>operation-failed</error-tag>
      <error-severity>error</error-severity>
      <source-daemon>
ifinfo
</source-daemon>
      <error-message>
device xe-0/0/0 not found
</error-message>
    </rpc-error>
  </interface-information>
</rpc-reply>

By the way, with ncclient, it is also possible to optionally output a text format that is easy for humans to see (= similar to the CLI result on the router).

show_junos.py


(abridgement)

# print connection.command('show interfaces terse xe-0/0/0')
print connection.command('show interfaces terse xe-0/0/0', format='text')

Execution result


$ python show_junos.py
1. run show command
========================================
<rpc-reply message-id="urn:uuid:xxxxxx-xxxxxx">
  <output>
Interface               Admin Link Proto    Local                 Remote
xe-0/0/0                down  down
</output>
</rpc-reply>

It is better to use XML-RPC format for processing by automation software, and text format for visual confirmation by the operator.

Acquisition of router config information

In ncclient, you can get the config information set in the current router with the get_config function and get function. In the example below, all the config information set in the router is acquired.

show_junos.py


#! /usr/bin/env python
# -*- coding: utf-8 -*-

from ncclient import manager

username = 'user1'
password = 'pass1'
ipv4 = '192.168.0.1'
port = 22

connection = manager.connect(host = ipv4, port = port, username = username, password = password, timeout = 20, device_params={'name':'junos'}, hostkey_verify=False )

print connection.get_config(source='running')

Execution result


$ python show_junos.py
<rpc-reply message-id="urn:uuid:xxxxxx-xxxxxx">
  <data>
    <configuration changed-seconds="1450593048" changed-localtime="2015-12-20 15:30:48 JST">
      <version>xx.xx.xx</version>
      <system>
        <host-name>router</host-name>
        <time-zone>Asia/Tokyo</time-zone>
      (Omitted below)

If you want to get the config partially, such as when you want to see the interface settings, you can get it by adding the filter option to the get_config function. However, since XML-RPC structure information is required at this time, it is necessary to check the XML-RPC schema in advance with the'show configration interaface xe-0 / 0/0 | display xml'command etc. ..

show_junos.py



(abridgement)

request_config_interface = """
<configuration>
    <interfaces>
        <interface>
            <name>xe-0/0/0</name>
        </interface>
    </interfaces>
</configuration>"""

# print connection.get_config(source='running')
print connection.get_config(source='running', filter=('subtree', request_config_interface) )

Execution result


$ python show_junos.py
1. run show command
========================================
<rpc-reply message-id="urn:uuid:xxxxxx-xxxxxx">
  <data>
    <configuration commit-seconds="1450593048" commit-localtime="2015-12-20 15:30:48 JST" commit-user="user1">
      <interfaces>
        <interface>
          <name>xe-0/0/0</name>
          <disable/>
        </interface>
      </interfaces>
    </configuration>
  </data>
</rpc-reply>

At this point, xe-0 / 0/0 contains only the'disable' setting.

CLI confirmation result on the router


user1@router> show configuration interfaces xe-0/0/0
disable;

Interface information settings

Now, let's create a program that inputs settings to the interface. The purpose here is to set the IP address shown below and enable the interface.

Settings you want to execute


delete interfaces xe-0/0/0 disable
set interfaces xe-0/0/0 unit 0 family inet address 10.0.0.1/30

To set the router config, use the edit_config function of ncclient. After dropping the above settings into the XML-RPC schema, specify it in the argument of edit_config and execute it.

Describe the created program. The flow of the program is as follows. --Step 1. Check the router configuration of the interface part in advance --Step 2. Set the router config using the edit_config function as the candidate config. --Step 3. Check the normality of the candidate config --Step 4. Display of candidate config --Step 5. Implementation of Commit (People judge here) --Step 6. Post-check the router config of the interface part

set_junos.py


#! /usr/bin/env python
# -*- coding: utf-8 -*-

from ncclient import manager

username = 'user1'
password = 'pass1'
ipv4 = '192.168.0.1'
port = 22

connection = manager.connect(host = ipv4, port = port, username = username, password = password, timeout = 20, device_params={'name':'junos'}, hostkey_verify=False )


print '='*40
print 'Step 1. show running-config before commit'
print '='*40

request_config_interface = """
<configuration>
    <interfaces>
        <interface>
            <name>xe-0/0/0</name>
        </interface>
    </interfaces>
</configuration>"""

print connection.get_config(source='running', filter=('subtree', request_config_interface) )


print '='*40
print 'Step 2. set config on candidate-config'
print '='*40

request_set_config_interface = """
<config>
    <configuration>
         <interfaces>
            <interface>
                <name>xe-0/0/0</name>
                <enable/>
                <unit>
                    <name>0</name>
                    <family>
                        <inet>
                            <address>
                                <name>10.0.0.1/30</name>
                            </address>
                        </inet>
                    </family>
                </unit>
            </interface>
        </interfaces>
    </configuration>
</config>
"""

print connection.edit_config(target='candidate', config=request_set_config_interface)


print '='*40
print 'Step 3. validate candidate-config'
print '='*40

print connection.validate(source='candidate')


print '='*40
print 'Step 4. show config on candicate-config'
print '='*40

print connection.get_config(source='candidate', filter=('subtree', request_config_interface) )


print '='*40
print 'Step 5. commit'
print '='*40
print 'Do you commit? y/n'
choice = raw_input().lower()
if choice == 'y':
    connection.commit()
else:
    connection.discard_changes()


print '='*40
print 'Step 6. show running-config after commit'
print '='*40

print connection.get_config(source='running', filter=('subtree', request_config_interface) )


if connection:
    connection.close_session()

When executed, it works as follows.

Execution result



$ python set_junos.py
========================================
Step 1. show running-config before commit
========================================
<rpc-reply message-id="urn:uuid:xxxxxx-xxxxxx">
  <data>
    <configuration commit-seconds="1450688416" commit-localtime="2015-12-21 18:00:16 JST" commit-user="user1">
      <interfaces>
        <interface>
          <name>xe-0/0/0</name>
          <disable/>
        </interface>
      </interfaces>
    </configuration>
  </data>
</rpc-reply>

========================================
Step 2. set config on candidate-config
========================================
<rpc-reply message-id="urn:uuid:xxxxxx-xxxxxx">
  <ok/>
</rpc-reply>

========================================
Step 3. validate candidate-config
========================================
<rpc-reply message-id="urn:uuid:xxxxxx-xxxxxx">
  <commit-results>
</commit-results>
  <ok/>
</rpc-reply>

========================================
Step 4. show config on candicate-config
========================================
<rpc-reply message-id="urn:uuid:xxxxxx-xxxxxx">
  <data>
    <configuration changed-seconds="1450688512" changed-localtime="2015-12-21 18:01:52 JST">
      <interfaces>
        <interface>
          <name>xe-0/0/0</name>
          <undocumented>
            <enable/>
          </undocumented>
          <unit>
            <name>0</name>
            <family>
              <inet>
                <address>
                  <name>10.0.0.1/30</name>
                </address>
              </inet>
            </family>
          </unit>
        </interface>
      </interfaces>
    </configuration>
  </data>
</rpc-reply>

========================================
Step 5. commit
========================================
Do you commit? y/n
y
========================================
Step 6. show running-config after commit
========================================
<rpc-reply message-id="urn:uuid:xxxxxx-xxxxxx">
  <data>
    <configuration commit-seconds="1450688535" commit-localtime="2015-12-21 18:02:15 JST" commit-user="user1">
      <interfaces>
        <interface>
          <name>xe-0/0/0</name>
          <undocumented>
            <enable/>
          </undocumented>
          <unit>
            <name>0</name>
            <family>
              <inet>
                <address>
                  <name>10.0.0.1/30</name>
                </address>
              </inet>
            </family>
          </unit>
        </interface>
      </interfaces>
    </configuration>
  </data>
</rpc-reply>

I was able to confirm that the settings were correct even with the router CLI.

CLI confirmation result on the router


user1@router> show configuration interfaces xe-0/0/0
enable;
unit 0 {
    family inet {
        address 10.0.0.1/30;
    }
}

What I felt when I made a NETCONF program

Software-friendly error output

The edit_config function executes the config setting, and if it succeeds, the result is output as follows.

<rpc-reply message-id="urn:uuid:xxxxxx-xxxxxx">
  <ok/>
</rpc-reply>

If the config setting fails, the error content will be output. (The example below is when trying to rewrite a setting that cannot be changed)

<rpc-reply message-id="urn:uuid:xxxxxx-xxxxxx">
  <rpc-error>
    <error-type>protocol</error-type>
    <error-tag>operation-failed</error-tag>
    <error-severity>error</error-severity>
    <error-message>
could not set enab_disab
</error-message>
  </rpc-error>
  <ok/>
</rpc-reply>

This is very helpful when developing automation programs. If the software uses the router CLI, it is awkward to say "Detect error message-> Execute show command to find problem part, parsing-> execute another show command, parsing-> ..." It was necessary to implement the software, but it is possible to move to the next process by extracting the error message by specifying only the necessary part with the XML tag, and the amount of programs can be significantly reduced. can.

The delete command doesn't work

I had a hard time implementing the delete command when creating the above program.

delete interfaces xe-0/0/0 disable;

Originally, in the XML-RPC schema, disable; is stored as follows.

<interfaces>
   <interface>
      <name>xe-0/0/0</name>
      <disable/>
   </interface>
</interfaces>

It was troublesome to remove this \ . This may be a problem with the implementation of ncclient, but "\ ... \ </ delete-config>" defined in RFC cannot be used with ncclient and an error occurs.

set_junos.py


(abridgement)

request_delete_config_interface = """
<delete-config>
    <interfaces>
       <interface>
           <name>xe-0/0/0</name>
           <disable/>
       </interface>
    </interfaces>
</delete-config>
"""

print connection.edit_config(target='candidate', config=request_delete_config_interface)

Execution result


Traceback (most recent call last):
  File "set_junos.py", line 62, in <module>
    print connection.edit_config(target='candidate', config=request_delete_config_interface)
  File "/usr/lib/python2.7/site-packages/ncclient/manager.py", line 157, in wrapper
    return self.execute(op_cls, *args, **kwds)
  File "/usr/lib/python2.7/site-packages/ncclient/manager.py", line 227, in execute
    raise_mode=self._raise_mode).request(*args, **kwds)
  File "/usr/lib/python2.7/site-packages/ncclient/operations/edit.py", line 62, in request
    node.append(validated_element(config, ("config", qualify("config"))))
  File "/usr/lib/python2.7/site-packages/ncclient/xml_.py", line 117, in validated_element
    raise XMLError("Element [%s] does not meet requirement" % ele.tag)
ncclient.xml_.XMLError: Element [delete-config] does not meet requirement

Next, refer to ncclient sample code and refer to "delete =" delete = "delete" in the XML tag. I also tried the description "", but although this works, it seems to be better to avoid it because it returns an RPC error response.

set_junos.py


(abridgement)

request_delete_config_interface = """
<config>
    <configuration>
         <interfaces>
            <interface delete="delete">
                <name>xe-1/3/0</name>
                <disable/>
            </interface>
        </interfaces>
    </configuration>
</config>
"""

print connection.edit_config(target='candidate', config=request_delete_config_interface)

Execution result


<rpc-reply message-id="urn:uuid:xxxxxx-xxxxxx">
  <rpc-error>
    <error-type>protocol</error-type>
    <error-tag>operation-failed</error-tag>
    <error-severity>error</error-severity>
    <error-message>
could not set enab_disab
</error-message>
  </rpc-error>
  <ok/>
</rpc-reply>

Finally, this time, I avoided (?) By setting "set enable;" instead of "delete disable;". I don't think it's a fundamental solution, but for the time being, I can now set it without any errors.


#### **`set_junos.py`**
```py

 (abridgement)

request_delete_config_interface = """
<config>
    <configuration>
         <interfaces>
            <interface>
                <name>xe-1/3/0</name>
                <enable/>
            </interface>
        </interfaces>
    </configuration>
</config>
"""

print connection.edit_config(target='candidate', config=request_delete_config_interface)

Execution result


<rpc-reply message-id="urn:uuid:xxxxxx-xxxxxx">
  <ok/>
</rpc-reply>

XML with "enable" set-Looking at the RPC, "<undocumented>..<undocumented>The details are unknown because it is surrounded by the tag, but XML-It may be a setting that is not defined in RPC.

<interfaces>
    <interface>
      <name>xe-1/3/0</name>
      <undocumented>
        <enable/>
      </undocumented>
    </interface>
</interfaces>

I haven't checked it myself, but at the moment it seems that there are some things that can not be set even with NETCONF, so it seems better to check in advance whether the network function you want to automate can be set with NETCONF. ..

##XML by manufacturer-RPC schema is different and difficult to find Originally I intended to write a program for JUNOS and IOS XR in this blog, but I gave up because it took too long to examine each XML schema.

JANOG36 NETCONF/YANG presentation materialAs far as I can see, it seems to write as follows in IOS XR.

IOSXR_XML-RPC


<Configuration>	
    <InterfaceConfigurationTable>	
        <InterfaceConfiguration>	
            <Naming>	
                <Active>act</Active>	
                <InterfaceName Match="Loopback0"/>	
            </Naming>
            <InterfaceVirtual>true</InterfaceVirtual>	
            <IPV4Network>
                <Addresses>
                    <Primary>	
                        <Address>172.16.255.1</Address>	
                        <Netmask>255.255.255.255</Netmask>	
                    </Primary>
                </Addresses>
            </IPV4Network>		
        </InterfaceConfiguration>	
    </InterfaceConfigurationTable>	
</Configuration>

Even if you compare it with the JUNOS XML schema used this time, you can see that it is quite different.

JUNOS_XML-RPC


<configuration>
      <interfaces>
        <interface>
          <name>xe-0/0/0</name>
          <undocumented>
            <enable/>
          </undocumented>
          <unit>
            <name>0</name>
            <family>
              <inet>
                <address>
                  <name>10.0.0.1/30</name>
                </address>
              </inet>
            </family>
          </unit>
        </interface>
      </interfaces>
    </configuration>

In the case of JUNOS, it is quite easy because there is a "display xml rpc" option, but it is a little painful because it is implemented while looking at the document to check the XML structure of other manufacturers.

We are happy that the industry standard protocol called NETCONF has come out, but at present it is still necessary to write a dedicated program for each router OS. The data model YANG is currently being actively discussed and implemented by the IETF, and the data structure of routers may soon be unified.

#Summary of using NETCONF I have summarized the advantages and disadvantages of using NETCONF at the moment. ##advantage -Status check result and error result are XML-Because it is structured by RPC, the amount of code for parsing and exception handling can be significantly reduced when creating automation software.

##Disadvantage -The XML structure differs depending on the network device manufacturer, and a dedicated program will be required for each router OS for a while. -XML for each manufacturer-It's hard to check RPC -Are there any features that cannot be set with NETCONF at this time? -I haven't found it myself, but I've heard that LACP and other settings couldn't be set.

#Finally It was a very good study for me to write this article. When writing a NETCONF program, more than writing a program, XML published by the manufacturer-You need to read a lot of RPC schema information and documentation for the libraries you use. At the moment, there are few sample codes, so it may be difficult.

User(=Network operator, software developer)XML for each manufacturer-RPCスキーマを意識させないUserフレンドリなラッパーAPIが登場すると実装の敷居がかなり下がるので、ネットワーク運用自動化が一気に進むように思います。 I introduced it on my blog beforeNAPALMIf an API library that allows devices from multiple vendors to be set with the same function, such as, will appear in the future, software implementation will be easier.

Past article: I tried touching the router control API library NAPALM (By the way, in NAPALM, it seems that only JUNOS is NETCONF, and the rest is SSH, vendor API, and various implementations.)

That's all for this article. We hope you find it useful when you start NETCONF. Thank you for your relationship.

Recommended Posts

Try setting NETCONF (ncclient) from software to the router
Try setting SSH (Exscript) from the software to the router
Try to make PC setting change software using TKinter (beginner)
[Python] Try to graph from the image of Ring Fit [OCR]
Try to introduce the theme to Pelican
Cython to try in the shortest
The fastest way to try EfficientNet
From "drawing" to "writing" the configuration diagram: Try drawing the AWS configuration diagram with Diagrams
The easiest way to try PyQtGraph
Try using the Python web framework Django (1)-From installation to server startup
Try to face the integration by parts
How to operate Linux from the console
Python amateurs try to summarize the list ①
How to access the Datastore from the outside
Try to write code from 1 using the machine learning framework chainer (mnist edition)
Let's try TensorFlow music generation project "Magenta" from development environment setting to song generation.
Try to solve the fizzbuzz problem with Keras
Try adding fisheye lens distortion to the image
Try to decompose the daimyo procession into Tucker
Try to solve the Python class inheritance problem
Setting to output the log of cron execution
Try to solve the man-machine chart with Python
How to try the friends-of-friends algorithm with pyfof
Change the decimal point of logging from, to.
How to operate Linux from the outside Procedure
POST images from ESP32-CAM (MicroPython) to the server
How to measure line speed from the terminal
From the introduction of pyethapp to the execution of contract
Try accessing the YQL API directly from Python 3
Try to simulate the movement of the solar system
The story of moving from Pipenv to Poetry
Try posting to Qiita for the first time
Try to create a battle record table with matplotlib from the data of "Schedule-kun"
Give latitude and longitude point sequence data and try to identify the road from OpenStreetMap data