I tried to graph the packages installed in Python

result

-> PyPI Top 100 Package Version: http://qiita.com/kitsuyui/items/a7cc56d476b59ff07f54

I got this figure with my virtualenv for machine learning. It's obvious at a glance.

dependencies.gv.png

I thought numpy and scipy were just as dependent, scipy is less dependent ...

The maintainer's struggle is exuded by the fact that many packages use six to make Python 3 and Python 2 compatible.

In the rest of the sections, pip freeze and pipdeptree, the script that outputs this figure is itemized.

pip freeze

python


$ pip freeze -l

Then you can see the packages in the current virtualenv. However, since there is no hierarchy and everything is described flat, it comes out in a row and I do not understand the dependency.

autoflake==0.6.6
autopep8==1.1.1
flake8==2.2.4
graphviz==0.4.3
hacking==0.10.1
isort==3.9.6
matplotlib==1.4.3
mccabe==0.2.1
natsort==3.5.6
nose==1.3.6
numpy==1.9.2
pandas==0.16.0
pbr==0.10.8
pep8==1.5.7
pies==2.6.3
pipdeptree==0.4.2
py==1.4.26
pyflakes==0.8.1
pyparsing==2.0.3
pytest==2.7.0
python-dateutil==2.4.2
pytz==2015.2
scipy==0.15.1
six==1.9.0
Theano==0.7.0
tox==1.9.2
virtualenv==12.1.1

Use pipdeptree

On the other hand, pipdeptree can be used to display the dependency hierarchy in an easy-to-understand manner.

python


$ pipdeptree --nowarn --local-only --freeze --all
autoflake==0.6.6
  - pyflakes==0.8.1
autopep8==1.1.1
  - pep8==1.5.7
flake8==2.2.4
  - pyflakes==0.8.1
  - pep8==1.5.7
  - mccabe==0.2.1
graphviz==0.4.3
hacking==0.10.1
  - pyflakes==0.8.1
  - six==1.9.0
  - flake8==2.2.4
    - pyflakes==0.8.1
    - pep8==1.5.7
    - mccabe==0.2.1
  - pep8==1.5.7
  - mccabe==0.2.1
  - pbr==0.10.8
    - pip
isort==3.9.6
  - pies==2.6.3
  - natsort==3.5.6
matplotlib==1.4.3
  - nose==1.3.6
  - python-dateutil==2.4.2
    - six==1.9.0
  - pyparsing==2.0.3
  - six==1.9.0
  - pytz==2015.2
  - numpy==1.9.2
mccabe==0.2.1
natsort==3.5.6
nose==1.3.6
numpy==1.9.2
pandas==0.16.0
  - pytz==2015.2
  - python-dateutil==2.4.2
    - six==1.9.0
  - numpy==1.9.2
pbr==0.10.8
  - pip
pep8==1.5.7
pies==2.6.3
py==1.4.26
pyflakes==0.8.1
pyparsing==2.0.3
pytest==2.7.0
  - py==1.4.26
python-dateutil==2.4.2
  - six==1.9.0
pytz==2015.2
scipy==0.15.1
six==1.9.0
Theano==0.7.0
  - numpy==1.9.2
  - scipy==0.15.1
tox==1.9.2
  - virtualenv==12.1.1
  - py==1.4.26
virtualenv==12.1.1

Let's make a graph.

I decided to use graphviz. The following is a script that passes the output of pipdeptree as standard input and graphs it with graphviz. Like my other articles, I use Python 3.

graphout.py


# coding: utf-8
import re

import graphviz


def depfile_to_tree(file_like_object):
    line_pattern = re.compile(r'^(?:(?P<indent>(?:  )*)- )?'
                              r'(?P<pkg_name>.*?)'
                              r'(?:==(?P<version>.*?))?\n$')

    tree = []

    for line in file_like_object:
        matcher = line_pattern.match(line)
        indent, pkg_name, version = matcher.group('indent',
                                                  'pkg_name',
                                                  'version')

        if indent is None:
            hierarchy = 0
        else:
            hierarchy = int(len(indent) / 2)

        if hierarchy < len(tree):
            tree = tree[:hierarchy]

        tree.append((pkg_name, version))
        yield tuple(tree)


def main(pipdeptree_file):
    packages = set()
    dependencies = set()

    for package_tree in depfile_to_tree(pipdeptree_file):
        pkg_name = package_tree[-1][0]
        packages.add(pkg_name)
        if len(package_tree) < 2:
            continue
        parent, child = package_tree[-2:]
        dependencies.add((parent[0], child[0], child[1]))

    dot = graphviz.Digraph(comment='My Pip Dependencies')
    for p in packages:
        dot.node(p)

    for d in dependencies:
        dot.edge(*d)

    dot.engine = 'dot'
    dot.format = 'png'
    dot.render('dependencies.gv')

if __name__ == '__main__':
    import sys
    main(sys.stdin)

How to use

python


$ pipdeptree --nowarn --local-only --freeze --all > dependencies.txt
$ python graphout.py < dependencies.txt

Generate dependencies.gv and dependencies.gv.png.

Digression

It seems that pip does not resolve true dependencies. It may be solved by using pip-tools (pip-review) or depsolver. Absent.

Recommended Posts

I tried to graph the packages installed in Python
I tried with the top 100 PyPI packages> I tried to graph the packages installed on Python
[Python] I tried to graph the top 10 eyeshadow rankings
I tried to implement the mail sending function in Python
I tried to implement PLSA in Python
I tried to implement permutation in Python
I tried to implement PLSA in Python 2
I tried to implement ADALINE in Python
I tried to implement PPO in Python
I tried simulating the "birthday paradox" in Python
I tried the least squares method in Python
I tried to implement TOPIC MODEL in Python
I tried to implement selection sort in python
I want to display the progress in Python!
[Python] I tried to summarize the set type (set) in an easy-to-understand manner.
I tried to display the altitude value of DTM in a graph
I tried to touch the CSV file with Python
I tried to solve the soma cube with python
I tried to implement a pseudo pachislot in Python
I tried to implement Dragon Quest poker in Python
I tried to implement GA (genetic algorithm) in Python
I want to write in Python! (3) Utilize the mock
I tried to summarize how to use pandas in python
I tried to solve the problem with Python Vol.1
I want to use the R dataset in python
I tried to complement the knowledge graph using OpenKE
Python OpenCV tried to display the image in text.
I tried to summarize the string operations of Python
I tried to touch Python (installation)
In the python command python points to python3.8
I wrote the queue in Python
I tried Line notification in Python
I tried to move the ball
I wrote the stack in Python
I tried to estimate the interval.
I tried to find the entropy of the image with python
I tried to simulate how the infection spreads with Python
I tried the accuracy of three Stirling's approximations in python
I tried to create API list.csv in Python from swagger.yaml
I tried to implement a one-dimensional cellular automaton in Python
I tried to summarize the code often used in Pandas
I tried "How to get a method decorated in Python"
I tried to illustrate the time and time in C language
I tried programming the chi-square test in Python and Java.
[Python] I tried to visualize the follow relationship of Twitter
I tried to summarize the commands often used in business
I tried to enumerate the differences between java and python
I tried to make a stopwatch using tkinter in python
I tried changing the python script from 2.7.11 to 3.6.0 on windows10
I tried to divide the file into folders with Python
I tried to implement blackjack of card game in Python
I tried to summarize Python exception handling
I tried to describe the traffic in real time with WebSocket
I tried to solve the ant book beginner's edition with python
I tried to summarize what python strong people are doing in the competition professional neighborhood
I tried to summarize the umask command
Movement that changes direction in the coordinate system I tried Python 3
I tried to recognize the wake word
I installed Python 3.5.1 to study machine learning
Python3 standard input I tried to summarize
I tried using Bayesian Optimization in Python