I tried to automatically generate a port management table from Config of L2SW

Introduction

In the previous article, I introduced an example of automatically generating Config from the port management table. I tried to automatically generate L2SW Config from port management table + parameter table + Jinja2 template

In some cases, you may want to create a port management table from the created Config. This time, I tried to automatically generate a CSV format port management table from the L2SW Config file using a Python parser library called TTP (Template Text Parser).

TTP overview

To introduce a simple example of Quick start in the document as it is, "(1) Config" is changed to "(2) Template". It is an image generated by "(3) Output result (JSON or CSV format)", which is the part of the character string enclosed in {{~}} after parsing using.

(1) Config

interface Loopback0
 description Router-id-loopback
 ip address 192.168.0.113/24
!
interface Vlan778
 description CPE_Acces_Vlan
 ip address 2002::fd37/124
 ip vrf CPE1
!

(2) Template

interface {{ interface }}
 ip address {{ ip }}/{{ mask }}
 description {{ description }}
 ip vrf {{ vrf }}

(3) Output result (JSON format)

python


[
    [
        {
            "description": "Router-id-loopback",
            "interface": "Loopback0",
            "ip": "192.168.0.113",
            "mask": "24"
        },
        {
            "description": "CPE_Acces_Vlan",
            "interface": "Vlan778",
            "ip": "2002::fd37",
            "mask": "124",
            "vrf": "CPE1"
        }
    ]
]

Config used

This time, I used almost the same Config file as the article mentioned at the beginning. Please refer to the GitHub repository below for details. Config file-config_hqaccess1.txt

Python code

The general flow is as follows.

  1. Perth Use parse_config () to output the parse result in JSON format from Config and template.

  2. Generate port management table Convert JSON to dictionary format and convert L2 interface parsing result to CSV format port management table with write_dict_to_csv ().

We will introduce each result in the following items.

portlist_generation.py


# -*- coding: utf-8 -*-
from ttp import ttp
import json
import csv

#Define paths for various files
TEMPLATE = './catalyst2960_template_ttp.txt'
PORT_LIST = './port_list_hqaccess1_ttp.csv'
CONFIG_FILENAME = './config_hqaccess1.txt'
CSV_COLUMNS = ['port_no', 'speed', 'duplex', 'mode', 'vlan', 'portfast', 'status', 'description']


def parse_config(template_file, config_filename):
    with open(config_filename, 'rt') as fc:
        data_to_parse = fc.read()

    with open(template_file, 'rt') as ft:
        ttp_template = ft.read()

    # create parser object and parse data using template:
    parser = ttp(data=data_to_parse, template=ttp_template)
    parser.parse()

    # print result in JSON format
    results = parser.result(format='json')[0]
    return results


def write_dict_to_csv(port_list, csv_columns, results):
    with open(port_list, 'w', newline='') as csvfile:   #For Windows, newline=''Is necessary
        writer = csv.DictWriter(csvfile, fieldnames=csv_columns)
        writer.writeheader()
        for data in results:
            writer.writerow(data)
    return            


def main():
    results = parse_config(TEMPLATE, CONFIG_FILENAME)
    print(results)
    results_dict = json.loads(results)
    write_dict_to_csv(PORT_LIST, CSV_COLUMNS, results_dict[0]['l2_interfaces'])


if __name__ == "__main__":
    main()

Templates and perspective results

I actually created one file (catalyst2960_template_ttp.txt), but for the sake of explanation, I made three for each setting item. It is divided.

(1) Global settings

Since the interface-related template was created in (2) and (3), other global settings are defined here.

part1_template


<group name="global_settings">
hostname {{ hostname }}
enable secret {{ secret }}
username {{ username }} privilege 15 password {{ password }}
ip domain-name {{ hostname }}
ip default-gateway {{ default_gw }}
ntp server {{ ntp_server }}
</group>

The setting items are divided into groups by <group> </ group>. The group name is global_settings. As a result, when the parsed result is output in JSON format, the result will be displayed in " global_settings ": {~}. I think it is convenient if you want to make it easier to see and separate the processing for each setting item.

Also, in the group, the part you want to parse is specified by {{~}} in a form similar to the Jinja2 template. The output result is as follows.

part1_output_json


[
    {
        "global_settings": {
            "default_gw": "192.168.100.150",
            "hostname": "hqaccess1",
            "ntp_server": "192.168.100.44",
            "password": "cisco",
            "secret": "test",
            "username": "test"
        }
    }
]

(2) VLAN interface settings

I created the following template with the group name vlan_interfaces.

part2_template


<macro>
def check_port_status(data):
    if "down" in data["port_status"]:
        data["status"] = "x"
    else:
        data["status"] = "o"
    return data
</macro>

<group name="vlan_interfaces" macro="check_port_status" del="port_status">
interface Vlan{{ vlan_num }}
 description {{ vlan_desc | ORPHRASE }}
 ip address {{ ip_address }} {{ subnet }}
 shut{{ port_status | default("up") }}
!{{ _end_ }}
</group>

In addition to (1), we are using four additional functions.

  1. Regular expression pattern ORPHRASE If the Description has one word, specify the regular expression pattern WORD. If there are multiple words separated by spaces, such as << To hqdist1 Gi0 / 1 >>, you can specify PHRASE to get to the end of the line. If you can take both on a case-by-case basis, specify ʻORPHRASE` as in this case.

  2. Specifying the default value default () Since the command following shut is targeted for parsing, if there is a shutdown command, the value of port_status will be down (blocked state). If there is no command, it defaults to ʻup` (open state).

  3. Group functions macro TTP allows you to set macros using Python code. Here, create the macro check_port_status, and if the value of port_status is down, specify the value of the newly created key status in x. For ʻup, use ʻo.

  4. Group Functions del Since only status is required for the open / blocked status information, port_status is deleted from the output result.

The output result is as follows.

part2_output_json


[
    {
        "vlan_interfaces": [
            {
                "status": "x",
                "vlan_num": "1"
            },
            {
                "ip_address": "192.168.100.47",
                "status": "o",
                "subnet": "255.255.255.0",
                "vlan_desc": "<< Server Segment >>",
                "vlan_num": "100"
            }
        ]
    }
]

(3) L2 interface setting

Finally, I created the following template with the group name l2_interfaces.

part3_template


<macro>
def check_port_status(data):
    if "down" in data["port_status"]:
        data["status"] = "x"
    else:
        data["status"] = "o"
    return data

def check_stp_option(data):
    if "portfast" in data["stp_option"]:
        data["portfast"] = "o"
    else:
        data["portfast"] = "x"
    return data
</macro>

<group name="l2_interfaces" exclude="ip_setting, no_ip_setting" macro="check_port_status, check_stp_option" del="port_status, stp_option">
interface {{ port_no }}
 description {{ description | ORPHRASE }}
 switchport access vlan {{ vlan | default("1") }}
 switchport trunk allowed vlan {{ vlan }}
 switchport mode {{ mode | default("access") }}
 duplex {{ duplex | default("auto") }}
 speed {{ speed | default("auto") }}
 shut{{ port_status | default("up") }}
 spanning-tree {{ stp_option | default("none") }}
 ip {{ ip_setting | ORPHRASE }}
 no ip {{ no_ip_setting | ORPHRASE }}
!{{ _end_ }}
</group>

Basically, it is an application of (1) and (2), but in order to output the result to the port management table only to the L2 interface, the templates ʻip {{ip_setting}}andno ip {{no_ip_setting}}` are used. If you use it and there is an IP setting (= L3 interface setting) that matches this, use the group function exclude. , The result of the corresponding interface is excluded.

The output result is as follows.

part3_output_json


[
    {
        "l2_interfaces": [
            {
                "description": "<< To PC1 >>",
                "duplex": "auto",
                "mode": "access",
                "port_no": "FastEthernet0/1",
                "portfast": "o",
                "speed": "auto",
                "status": "o",
                "vlan": "100"
            },
            {
                "description": "<< To PC2 >>",
                "duplex": "auto",
                "mode": "access",
                "port_no": "FastEthernet0/2",
                "portfast": "o",
                "speed": "auto",
                "status": "o",
                "vlan": "100"
            },
            {
                "duplex": "auto",
                "mode": "access",
                "port_no": "FastEthernet0/3",
                "portfast": "o",
                "speed": "auto",
                "status": "x",
                "vlan": "100"
            },
            {
                "duplex": "auto",
                "mode": "access",
                "port_no": "FastEthernet0/4",
                "portfast": "o",
                "speed": "auto",
                "status": "x",
                "vlan": "100"
            },
            {
                "description": "<< To PC3 >>",
                "duplex": "auto",
                "mode": "access",
                "port_no": "FastEthernet0/5",
                "portfast": "o",
                "speed": "auto",
                "status": "o",
                "vlan": "101"
            },
            {
                "description": "<< To PC4 >>",
                "duplex": "auto",
                "mode": "access",
                "port_no": "FastEthernet0/6",
                "portfast": "o",
                "speed": "auto",
                "status": "o",
                "vlan": "101"
            },
            {
                "duplex": "auto",
                "mode": "access",
                "port_no": "FastEthernet0/7",
                "portfast": "o",
                "speed": "auto",
                "status": "x",
                "vlan": "101"
            },
            {
                "duplex": "auto",
                "mode": "access",
                "port_no": "FastEthernet0/8",
                "portfast": "o",
                "speed": "auto",
                "status": "x",
                "vlan": "101"
            },
            {
                "description": "<< To hqdist1 Gi0/1 >>",
                "duplex": "full",
                "mode": "trunk",
                "port_no": "GigabitEthernet0/1",
                "portfast": "x",
                "speed": "1000",
                "status": "o",
                "vlan": "100-101"
            },
            {
                "description": "<< To hqdist2 Gi0/1 >>",
                "duplex": "full",
                "mode": "trunk",
                "port_no": "GigabitEthernet0/2",
                "portfast": "x",
                "speed": "1000",
                "status": "o",
                "vlan": "100-101"
            }
        ]
    }
]

Generate port management table

The output result (3) in the previous section is converted to dictionary format, and the result output in CSV is as follows. portlist_output.png

[Last port management table](https://qiita.com/tech_kitara/items/b6ffd9790483b08b568b#%E3%83%9D%E3%83%BC%E3%83%88%E7%AE%A1%E7%90 Compared to% 86% E8% A1% A8), there is a difference between cells E10 and E11 due to the addition of the command "switchport trunk allowed vlan 100-101" to Gi0 / 1 and Gi0 / 2, but otherwise basically The same thing was generated.

Finally

Until now, I used TextFSM and Genie Parser as parsers for NW devices, but if you want to create your own parser that suits your environment, I think TTP is also a powerful option in terms of functionality and customizability. The set of files introduced this time has been uploaded to GitHub --portlist_generator, so I hope you find it helpful.

Recommended Posts

I tried to automatically generate a port management table from Config of L2SW
I tried to automatically generate a password with Python3
I made a tool to automatically generate a simple ER diagram from the CREATE TABLE statement
I tried using PI Fu to generate a 3D model of a person from one image
I made a plugin to generate Markdown table from csv in Vim
I tried to generate a random character string
I tried to create a table only with Django
I tried to automatically generate a character string to be input to Mr. Adjustment with Python
[Python] I tried to automatically create a daily report of YWT with Outlook mail
I want to start a lot of processes from python
I want to automatically generate a modern metal band name
I made a command to generate a table comment in Django
I tried to get a database of horse racing using Pandas
A memorandum when I tried to get it automatically with selenium
I tried to make a regular expression of "amount" using Python
I tried to make a regular expression of "time" using Python
I tried to create a list of prime numbers with python
I tried to make a regular expression of "date" using Python
I tried to get a list of AMI Names using Boto3
I tried to cut out a still image from the video
I tried to automatically collect images of Kanna Hashimoto with Python! !!
I tried to make a mechanism of exclusive control with Go
I tried to create a linebot (implementation)
I tried to create a linebot (preparation)
A story I was addicted to when inserting from Python to a PostgreSQL table
I tried to automatically extract the movements of PES players with software
I tried to send a registration completion email from Gmail with django.
[Outlook] I tried to automatically create a daily report email with Python
I tried to make a Web API
I tried to get the RSS of the top song of the iTunes store automatically
I tried to perform a cluster analysis of customers using purchasing data
I tried to display the altitude value of DTM in a graph
I tried to verify the result of A / B test by chi-square test
I tried 3D detection of a car
I tried to implement a card game of playing cards in Python
I tried to make a simple mail sending application with tkinter of Python
[Azure] I tried to create a Linux virtual machine in Azure of Microsoft Learn
[IBM Cloud] I tried to access the Db2 on Cloud table from Cloud Funtions (python)
I tried to extract a line art from an image with Deep Learning
I made a tool to generate Markdown from the exported Scrapbox JSON file
I made a tool to automatically back up the metadata of the Salesforce organization
[Python] I tried to get the type name as a string from the type function
I tried to create a model with the sample of Amazon SageMaker Autopilot
I tried to automatically send the literature of the new coronavirus to LINE with Python
I tried to build a super-resolution method / ESPCN
I tried the pivot table function of pandas
I implemented DCGAN and tried to generate apples
I tried to notify slack of Redmine update
I tried to find 100 million digits of pi
I tried to touch the API of ebay
How to generate a Python object from JSON
[Memo] I tried a pivot table in Python
I tried to build a super-resolution method / SRCNN ③
I tried to build a super-resolution method / SRCNN ②
I tried to make a ○ ✕ game using TensorFlow
I tried to predict the price of ETF
I tried to vectorize the lyrics of Hinatazaka46!
I tried to make a function to retrieve data from database column by column using sql with sqlite3 of python [sqlite3, sql, pandas]
I tried to make something like a chatbot with the Seq2Seq model of TensorFlow
I tried to notify the update of "Become a novelist" using "IFTTT" and "Become a novelist API"
Try to create a battle record table with matplotlib from the data of "Schedule-kun"