From "drawing" to "writing" the configuration diagram: Try drawing the AWS configuration diagram with Diagrams

things to do

Try drawing an AWS configuration diagram with code in Diagrams, also known as "Diagram as Code"

What are Diagrams

--Diagrams is a Python module --Graphviz is also required to use Diagrams --You can draw cloud system architecture with Python code --The site is here - https://diagrams.mingrammer.com/ ――The explanation of Diagrams on the site is in English, but the amount is small, so please read it! (Automatic translation is enough to understand)

In this way, you can write a configuration diagram with Python code. When this code is executed as a Python script, the configuration diagram is output as an image.

image.png

-** Good points ** --You can code the configuration diagram ――Since you can edit the drawing by code editing, it is easier than image correction --Since it is a code, it can be managed with Git together with the CloudFormation code. --Non-AWS icons are also available --The output configuration diagram can be used as a rough sketch or better.

Diagrams installation environment

I installed Diagrams in two environments, " Raspberry Pi 4 Model B " and " Cloud9 ".

Environment (1) Raspberry Pi 4 Model B

$ cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

Python comes in two versions, 2.7 and 3.7. Install Diagrams on Python 3.7

$ python --version
Python 2.7.16
$ python3 --version
Python 3.7.3

Environment (2) Cloud9

$ cat /etc/os-release
NAME="Amazon Linux AMI"
VERSION="2018.03"
ID="amzn"
ID_LIKE="rhel fedora"
VERSION_ID="2018.03"
PRETTY_NAME="Amazon Linux AMI 2018.03"
ANSI_COLOR="0;33"
CPE_NAME="cpe:/o:amazon:linux:2018.03:ga"
HOME_URL="http://aws.amazon.com/amazon-linux-ami/"

Cloud9 is Python 3.x only, not Python 2.x

$ python --version
Python 3.6.8

Diagrams installation

Installation (1) Raspberry Pi

Just install diagrams and Graphviz on Python3.

Install diagram on python3

$ python3 -m pip install diagrams

Graphviz installation

$ sudo apt-get install graphviz

Installation (2) Cloud9

$ sudo pip install --upgrade pip
$ sudo python3 -m pip install diagrams
$ sudo yum install graphviz

(Verification) Try drawing the AWS configuration diagram

(Verification 1) Draw a configuration diagram of the Web server

Here's what I was able to do, a decent degree of perfection

図1.png

I could see it like this on Cloud9 (convenient!)

image.png

--Outline of configuration --Build a WebApp server on EC2 --Prepare two WebApp servers for redundancy --Distribute the load on the WebApp server with ALB --Domain registration with Rote53 --Prepare WebApp server database with RDS

This is the code I made.

Web server configuration diagram


from diagrams import Cluster, Diagram, Edge
from diagrams.aws.network import VPC
from diagrams.aws.compute import EC2
from diagrams.aws.database import RDS
from diagrams.aws.network import Route53
from diagrams.aws.network import NATGateway
from diagrams.aws.network import InternetGateway
from diagrams.aws.network import ElasticLoadBalancing
from diagrams.onprem.network import Internet
from diagrams.onprem.client import Client
from diagrams.onprem.client import Users

with Diagram("Fig-001-WebSite", show=False):
    with Cluster("Fig-001-WebSite"):
        internet_01 = Internet("Internet")
        users_01 = Users("user")
        with Cluster("AWS/Develop/ap-northeast-1"):
            r53_01 = Route53("hoge.fuga.local")
            with Cluster("VPC/192.168.0.0/HOGE-SYSTEM"):
                alb_01 = ElasticLoadBalancing("ALB")
                igw_01 = InternetGateway("InternetGateway")
                with Cluster("subnet/192.168.1.0/public"):
                    natgw_01 = NATGateway("NATGW")
                    ec2_01 = EC2("WebApp")
                with Cluster("subnet/192.168.101.0/private"):
                    rds_01 = RDS("postgres-Master")
                with Cluster("subnet/192.168.2.0/public"):
                    natgw_02 = NATGateway("NATGW")
                    ec2_02 = EC2("WebApp")
                with Cluster("subnet/192.168.102.0/private"):
                    rds_02 = RDS("postgres-Slave")

    #Configuration diagram Web server
    users_01 - internet_01 - igw_01 - alb_01 #User access route
    alb_01 - ec2_01 - rds_01 #WebApp redundancy
    alb_01 - ec2_02 - rds_01 #WebApp redundancy
    rds_01 - Edge(style="dotted") - rds_02 #DB redundancy

(Verification 2) Draw a block diagram of WorkSpaces

--Amazon WorkSpaces is a virtual desktop service provided by AWS. --Diagrams does not have a WorkSpaces icon --This is the configuration diagram you want to create ↓

image.png

This is what I wrote in Diagrams (I think it's reasonable)

図2.png

--Icons that are not in Diagrams such as Storage Gateway and WorkSpaces are replaced with different icons.

Client icons are used instead of icons that are not provided in Diagrams, such as WorkSpaces and ENI.

WorkSpace configuration diagram


from diagrams import Cluster, Diagram, Edge
from diagrams.aws.network import VPC
from diagrams.aws.compute import EC2
from diagrams.aws.database import RDS
from diagrams.aws.network import Route53
from diagrams.aws.network import NATGateway
from diagrams.aws.network import InternetGateway
from diagrams.onprem.network import Internet
from diagrams.onprem.client import Client
from diagrams.aws.security import DirectoryService
from diagrams.aws.network import DirectConnect
from diagrams.onprem.compute import Server
from diagrams.onprem.client import User
from diagrams.onprem.compute import Nomad

with Diagram("Fig-002-WorkSpaces", show=False):
    with Cluster("Fig-002-WorkSpaces"):
        Internet_01 = Internet("Internet")
        User_01 = User("User 1")
        User_02 = User("User 2")
        UserDevice_01 = Client("device")
        UserDevice_02 = Client("device")

        with Cluster("VPC/AWS management"):
            StorageGw_01 = Client("StorageGateway")
            WorkSpace_01 = Client("WorkSpacesForUser1")
            WorkSpace_02 = Client("WorkSpacesForUser2")

        with Cluster("VPC/192.168.0.0/HOGE"):
            with Cluster("subnet/192.168.1.0/public"):
                eni_01 = Client("ENI User1")
                eni_02 = Client("ENI User2")

        DirectConnect_01 = DirectConnect("DirectConnect")
        ADConnector_01 = DirectoryService("AD Connector")

        with Cluster("Base"):
            with Cluster("Internal network"):
                ActiveDirectory_01 = Server("AD")

    #Configuration diagram WorkSpace
    User_01 - UserDevice_01 - Internet_01 - StorageGw_01 - WorkSpace_01 - eni_01 #device-Communication path between WorkSpaces
    User_02 - UserDevice_02 - Internet_01
    StorageGw_01 - WorkSpace_02 - eni_02
    ADConnector_01 - DirectConnect_01 - ActiveDirectory_01 #In-base AD-Communication path between AD Connector
    ADConnector_01 - Edge(style="dotted") - WorkSpace_01
    ADConnector_01 - Edge(style="dotted") - WorkSpace_02

Impressions: What I thought about using Diagrams

(* 1) If you decide the rules, you can create a composition diagram in your brain by looking at it. image.png

bonus

Put the icon output and Python code of onPrem (information as of May 2020)

onprem-aicons.png

onPrem icon output


from diagrams import Cluster, Diagram, Edge
# onprem.database
from diagrams.onprem.database import Cassandra
from diagrams.onprem.database import Clickhouse, ClickHouse
from diagrams.onprem.database import Cockroachdb, CockroachDB
from diagrams.onprem.database import Couchdb, CouchDB
from diagrams.onprem.database import Dgraph
from diagrams.onprem.database import Hbase, HBase
from diagrams.onprem.database import Influxdb, InfluxDB
from diagrams.onprem.database import Janusgraph, JanusGraph
from diagrams.onprem.database import Mariadb, MariaDB
from diagrams.onprem.database import Mongodb, MongoDB
from diagrams.onprem.database import Mssql, MSSQL
from diagrams.onprem.database import Mysql, MySQL
from diagrams.onprem.database import Neo4J
from diagrams.onprem.database import Oracle
from diagrams.onprem.database import Postgresql, PostgreSQL
from diagrams.onprem.database import Scylla
# onprem.security
from diagrams.onprem.security import Trivy
from diagrams.onprem.security import Vault
# onprem.ci
from diagrams.onprem.ci import Circleci, CircleCI
from diagrams.onprem.ci import Jenkins
from diagrams.onprem.ci import Teamcity, TC
from diagrams.onprem.ci import Travisci, TravisCI
# onprem.etl
from diagrams.onprem.etl import Embulk
# onprem.mlops
from diagrams.onprem.mlops import Polyaxon
# onprem.network
from diagrams.onprem.network import Apache
from diagrams.onprem.network import Caddy
from diagrams.onprem.network import Consul
from diagrams.onprem.network import Envoy
from diagrams.onprem.network import Etcd, ETCD
from diagrams.onprem.network import Haproxy, HAProxy
from diagrams.onprem.network import Internet
from diagrams.onprem.network import Istio
from diagrams.onprem.network import Kong
from diagrams.onprem.network import Linkerd
from diagrams.onprem.network import Nginx
from diagrams.onprem.network import Pfsense, PFSense
from diagrams.onprem.network import Pomerium
from diagrams.onprem.network import Tomcat
from diagrams.onprem.network import Traefik
from diagrams.onprem.network import Vyos, VyOS
from diagrams.onprem.network import Zookeeper
# onprem.inmemory
from diagrams.onprem.inmemory import Aerospike
from diagrams.onprem.inmemory import Hazelcast
from diagrams.onprem.inmemory import Memcached
from diagrams.onprem.inmemory import Redis
# onprem.search
from diagrams.onprem.search import Elasticsearch
from diagrams.onprem.search import Solr
# onprem.container
from diagrams.onprem.container import Docker
from diagrams.onprem.container import Rkt, RKT
# onprem.iac
from diagrams.onprem.iac import Ansible
from diagrams.onprem.iac import Awx
from diagrams.onprem.iac import Terraform
# onprem.compute
from diagrams.onprem.compute import Nomad
from diagrams.onprem.compute import Server
# onprem.vcs
from diagrams.onprem.vcs import Git
from diagrams.onprem.vcs import Github
from diagrams.onprem.vcs import Gitlab
# onprem.workflow
from diagrams.onprem.workflow import Airflow
from diagrams.onprem.workflow import Digdag
from diagrams.onprem.workflow import Kubeflow, KubeFlow
from diagrams.onprem.workflow import Nifi, NiFi
# onprem.queue
from diagrams.onprem.queue import Activemq, ActiveMQ
from diagrams.onprem.queue import Celery
from diagrams.onprem.queue import Kafka
from diagrams.onprem.queue import Rabbitmq, RabbitMQ
from diagrams.onprem.queue import Zeromq, ZeroMQ
# onprem.cd
from diagrams.onprem.cd import Spinnaker
# onprem.gitops
from diagrams.onprem.gitops import Argocd, ArgoCD
# onprem.monitoring
from diagrams.onprem.monitoring import Datadog
from diagrams.onprem.monitoring import Grafana
from diagrams.onprem.monitoring import Kibana
from diagrams.onprem.monitoring import Prometheus
from diagrams.onprem.monitoring import Splunk
from diagrams.onprem.monitoring import Thanos
# onprem.client
from diagrams.onprem.client import Client
from diagrams.onprem.client import User
from diagrams.onprem.client import Users
# onprem.logging
from diagrams.onprem.logging import Fluentd
from diagrams.onprem.logging import Logstash, LogStash
from diagrams.onprem.logging import Loki
# onprem.analytics
from diagrams.onprem.analytics import Beam
from diagrams.onprem.analytics import Flink
from diagrams.onprem.analytics import Hadoop
from diagrams.onprem.analytics import Hive
from diagrams.onprem.analytics import Metabase
from diagrams.onprem.analytics import Norikra
from diagrams.onprem.analytics import Spark
from diagrams.onprem.analytics import Storm
from diagrams.onprem.analytics import Tableau



with Diagram("OnPrem-Aicons", show=False):
    with Cluster("OnPrem"):
        # onprem.database("# onprem.database")
        Cassandra("Cassandra")
        Clickhouse, ClickHouse("Clickhouse, ClickHouse")
        Cockroachdb, CockroachDB("Cockroachdb, CockroachDB")
        Couchdb, CouchDB("Couchdb, CouchDB")
        Dgraph("Dgraph")
        Hbase, HBase("Hbase, HBase")
        Influxdb, InfluxDB("Influxdb, InfluxDB")
        Janusgraph, JanusGraph("Janusgraph, JanusGraph")
        Mariadb, MariaDB("Mariadb, MariaDB")
        Mongodb, MongoDB("Mongodb, MongoDB")
        Mssql, MSSQL("Mssql, MSSQL")
        Mysql, MySQL("Mysql, MySQL")
        Neo4J("Neo4J")
        Oracle("Oracle")
        Postgresql, PostgreSQL("Postgresql, PostgreSQL")
        Scylla("Scylla")
        # onprem.security("# onprem.security")
        Trivy("Trivy")
        Vault("Vault")
        # onprem.ci("# onprem.ci")
        Circleci, CircleCI("Circleci, CircleCI")
        Jenkins("Jenkins")
        Teamcity, TC("Teamcity, TC")
        Travisci, TravisCI("Travisci, TravisCI")
        # onprem.etl("# onprem.etl")
        Embulk("Embulk")
        # onprem.mlops("# onprem.mlops")
        Polyaxon("Polyaxon")
        # onprem.network("# onprem.network")
        Apache("Apache")
        Caddy("Caddy")
        Consul("Consul")
        Envoy("Envoy")
        Etcd, ETCD("Etcd, ETCD")
        Haproxy, HAProxy("Haproxy, HAProxy")
        Internet("Internet")
        Istio("Istio")
        Kong("Kong")
        Linkerd("Linkerd")
        Nginx("Nginx")
        Pfsense, PFSense("Pfsense, PFSense")
        Pomerium("Pomerium")
        Tomcat("Tomcat")
        Traefik("Traefik")
        Vyos, VyOS("Vyos, VyOS")
        Zookeeper("Zookeeper")
        # onprem.inmemory("# onprem.inmemory")
        Aerospike("Aerospike")
        Hazelcast("Hazelcast")
        Memcached("Memcached")
        Redis("Redis")
        # onprem.search("# onprem.search")
        Elasticsearch("Elasticsearch")
        Solr("Solr")
        # onprem.container("# onprem.container")
        Docker("Docker")
        Rkt, RKT("Rkt, RKT")
        # onprem.iac("# onprem.iac")
        Ansible("Ansible")
        Awx("Awx")
        Terraform("Terraform")
        # onprem.compute("# onprem.compute")
        Nomad("Nomad")
        Server("Server")
        # onprem.vcs("# onprem.vcs")
        Git("Git")
        Github("Github")
        Gitlab("Gitlab")
        # onprem.workflow("# onprem.workflow")
        Airflow("Airflow")
        Digdag("Digdag")
        Kubeflow, KubeFlow("Kubeflow, KubeFlow")
        Nifi, NiFi("Nifi, NiFi")
        # onprem.queue("# onprem.queue")
        Activemq, ActiveMQ("Activemq, ActiveMQ")
        Celery("Celery")
        Kafka("Kafka")
        Rabbitmq, RabbitMQ("Rabbitmq, RabbitMQ")
        Zeromq, ZeroMQ("Zeromq, ZeroMQ")
        # onprem.cd("# onprem.cd")
        Spinnaker("Spinnaker")
        # onprem.gitops("# onprem.gitops")
        Argocd, ArgoCD("Argocd, ArgoCD")
        # onprem.monitoring("# onprem.monitoring")
        Datadog("Datadog")
        Grafana("Grafana")
        Kibana("Kibana")
        Prometheus("Prometheus")
        Splunk("Splunk")
        Thanos("Thanos")
        # onprem.client("# onprem.client")
        Client("Client")
        User("User")
        Users("Users")
        # onprem.logging("# onprem.logging")
        Fluentd("Fluentd")
        Logstash, LogStash("Logstash, LogStash")
        Loki("Loki")
        # onprem.analytics("# onprem.analytics")
        Beam("Beam")
        Flink("Flink")
        Hadoop("Hadoop")
        Hive("Hive")
        Metabase("Metabase")
        Norikra("Norikra")
        Spark("Spark")
        Storm("Storm")
        Tableau("Tableau")

Recommended Posts

From "drawing" to "writing" the configuration diagram: Try drawing the AWS configuration diagram with Diagrams
I tried to draw a system configuration diagram with Diagrams on Docker
Try to solve the fizzbuzz problem with Keras
Try to solve the man-machine chart with Python
How to try the friends-of-friends algorithm with pyfof
Try to solve the programming challenge book with python3
Try setting SSH (Exscript) from the software to the router
Try setting NETCONF (ncclient) from software to the router
Try to create a battle record table with matplotlib from the data of "Schedule-kun"
Try to visualize the room with Raspberry Pi, part 1
Try to get the contents of Word with Golang
Try to specify the axis with PyTorch's Softmax function
I tried to draw a configuration diagram using Diagrams
Try to play with the uprobe that supports Systemtap directly
Getting Started with Drawing with matplotlib: Creating Diagrams from Data Files
[AWS] Try adding Python library to Layer with SAM + Lambda (Python)
Get the package version to register with PyPI from Git
Try to automate the operation of network devices with Python
Try to decipher the garbled attachment file name with Python
Try to extract the features of the sensor data with CNN
Try to factorial with recursion
Tokyo Corona: Try to make a simple prediction from open data with the exponential function curve_fit
Try to calculate the position of the transmitter from the radio wave propagation model with python [Wi-Fi, Beacon]
[Day 3] Study session to deploy from docker-compose to Kubernetes with minimal configuration
Repeat with While. Scripts to Tweet and search from the terminal
Create an alias for Route53 to CloudFront with the AWS API
[Python] Try to graph from the image of Ring Fit [OCR]
How to make a command to read the configuration file with pyramid
Try to solve the N Queens problem with SA of PyQUBO
Try to extract a character string from an image with Python3
From the introduction of JUMAN ++ to morphological analysis of Japanese with Python
Try to solve the shortest path with Python + NetworkX + social data
I tried Diagram as Code (Diagrams) which can manage the configuration diagram code
[Python] Try to recognize characters from images with OpenCV and pyocr
[Completed version] Try to find out the number of residents in the town from the address list with Python
Create folders from '01' to '12' with python
Try to operate Facebook with Python
Try to profile with ONNX Runtime
Try to introduce the theme to Pelican
Cython to try in the shortest
Try blurring the image with opencv2
Try to output audio with M5STACK
The fastest way to try EfficientNet
The easiest way to try PyQtGraph
Put Cabocha 0.68 on Windows and try to analyze the dependency with Python
Add 95% confidence intervals on both sides to the diagram with Python / Matplotlib
Try using the Python web framework Django (1)-From installation to server startup
Try to solve the traveling salesman problem with a genetic algorithm (Theory)
[Cloudian # 1] Try to access object storage with AWS SDK for Python (boto3)
I tried to learn the angle from sin and cos with chainer
Try to react only the carbon at the end of the chain with SMARTS
[Cloudian # 5] Try to list the objects stored in the bucket with Python (boto3)
Try to separate the background and moving object of the video with OpenCV
The story of migrating from home server (MariaDB + Java) to AWS (DynamoDB + Python + PHP) with reduced monthly cost
Try to measure the position of the object on the desk (real coordinate system) from the camera image with Python + OpenCV