I tried running the Python Package Repository (Warehouse) that supports PyPI

Overview

--PyPI has been migrating to the next generation repository implementation (Warehouse) since around 2018, and this source code is available on GitHub (https://github.com/pypa/warehouse). --A definition file for docker-compose is provided, and it is possible to start the development environment relatively easily. --Since it is possible to register a package from a client such as twine to an instance of warehouse, it can be used as a test environment for simple package operations. --PyPI provides a test instance (https://test.pypi.org/), but it is a server that is exposed to the outside. --In the case of warehouse, it can be used in a private environment & no account registration is required, so it is easy to use as a sandbox (may be limited to cases where you have the motivation to check how it looks on PyPI). I don't know).

Evaluation memo

Procedure memo when running on CentOS 7.7.

Premise

Install docker / docker-compose. If you look at requirements / main.in in the GitHub repository, it's appropriate (including ʻelasticsearch and postgresql`). It's a complicated stack, but it's basically resolved inside the repository.

Installation

The installation procedure is [Warehouse --Docs »Development» Getting started] (https://warehouse.readthedocs.io/development/getting-started/) explains.

After cloning the source code from github, execute the make build command to generate docker images.

$ git clone https://github.com/pypa/warehouse.git
$ cd warehouse
$ make build
$ docker images
REPOSITORY          TAG                  IMAGE ID            CREATED             SIZE
warehouse_web       latest               a78a164cbb8f        2 minutes ago       376MB
warehouse_worker    latest               a78a164cbb8f        2 minutes ago       376MB
<none>              <none>               5768f70bcf35        2 minutes ago       676MB
<none>              <none>               c34b08b4ed16        5 minutes ago       1.16GB
warehouse_static    latest               15277f26921d        6 minutes ago       1.14GB
python              3.7.3-slim-stretch   338ae06dfca5        6 months ago        143MB
node                8.15.1               8c51cec97ebf        9 months ago        895MB

Start the container group from the generated image with the following command.

$ make serve

After the static container is fully started, execute the following command (for example, in another terminal). (If you do not execute it, an Error will occur when accessing the Web GUI because the DB does not exist)

$ make initdb

When you access port 80 with a browser, you will see a screen that is almost the same as pypi. The main differences are as follows.

--The package contained in the repository contains test data (snapshot of test.pypi.org with private information removed). --pyramid_debugtoolbar is set and you can access debug information. --In addition to performance information, you can also refer to the REST API Route list.

Below is a sample of REST-API (Endpoint for getting package information) executed with curl.

$ yum install jq
$ curl -s http://127.0.0.1/pypi/logmmse/1.2/json | jq .info.name
"logmmse"

$ curl -s http://127.0.0.1/pypi/logmmse/json | jq '.releases|keys'
[
  "1.1",
  "1.2"
]

Package upload

You can register by simply switching the upload destination from a client such as twine. The following is an example of uploading the sample package generated by cookiecuter.

Install the prerequisite libraries.

$ yum install python36 -y
$ pip3 install twine wheel
$ pip3 install cookiecutter

Generate a sample package using a template (ʻaudreyr / cookiecutter-pypackage). I changed only project_slug` to avoid conflicts, but the others are defaults.

$ cookiecutter https://github.com/audreyr/cookiecutter-pypackage
full_name [Audrey Roy Greenfeld]:
email [[email protected]]:
github_username [audreyr]:
project_name [Python Boilerplate]:
project_slug [python_boilerplate]: python_boilerplate_1234
project_short_description [Python Boilerplate contains all the boilerplate you need to create a Python package.]:
pypi_username [audreyr]:
version [0.1.0]:
use_pytest [n]:
use_pypi_deployment_with_travis [y]:
add_pyup_badge [n]:
Select command_line_interface:
1 - Click
2 - Argparse
3 - No command-line interface
Choose from 1, 2, 3 [1]:
create_author_file [y]:
Select open_source_license:
1 - MIT license
2 - BSD license
3 - ISC license
4 - Apache Software License 2.0
5 - GNU General Public License v3
6 - Not open source
Choose from 1, 2, 3, 4, 5, 6 [1]:

Generate distribution of packages (source + binary). Confirm that the file is generated in the dist folder.

$ cd python_boilerplate_1234
$ python3 setup.py sdist
$ python3 setup.py bdist_wheel
$ ls dist
python_boilerplate_1234-0.1.0-py2.py3-none-any.whl  python_boilerplate_1234-0.1.0.tar.gz

Upload the package with twine upload.

--Specify the host of warehouse in --repository-url ( 127.0.0.1 in this example). --Use the warehouse test account (ʻewdurbin / password`) for the credentials required for uploading. - https://warehouse.readthedocs.io/development/getting-started/#logging-in-to-warehouse

$ twine upload --repository-url http://127.0.0.1/legacy/ dist/*
Uploading distributions to http://127.0.0.1/legacy/
Enter your username: ewdurbin
Enter your password:
Uploading python_boilerplate_1234-0.1.0-py2.py3-none-any.whl
100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 9.28k/9.28k [00:05<00:00, 1.73kB/s]
Uploading python_boilerplate_1234-0.1.0.tar.gz
100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 13.2k/13.2k [00:00<00:00, 25.6kB/s]

When the upload is completed normally, you can check the package from the GUI. You can also refer to the package information using REST-API.

$ curl -s http://127.0.0.1/pypi/python-boilerplate-1234/json | jq '.releases|keys'
[
  "0.1.0"
]

reference

Recommended Posts

I tried running the Python Package Repository (Warehouse) that supports PyPI
Miscellaneous notes that I tried using python for the matter
I tried Grumpy (Go running Python).
I tried running prolog with python 3.8.2.
Movement that changes direction in the coordinate system I tried Python 3
Python: I tried the traveling salesman problem
I tried the Python Tornado Testing Framework
I tried with the top 100 PyPI packages> I tried to graph the packages installed on Python
I tried "smoothing" the image with Python + OpenCV
[Python] I tried substituting the function name for the function name
I tried "differentiating" the image with Python + OpenCV
I tried simulating the "birthday paradox" in Python
I tried the least squares method in Python
I tried python programming for the first time.
I tried "binarizing" the image with Python + OpenCV
I tried running faiss with python, Go, Rust
I tried using the Datetime module by Python
I tried running python -m summpy.server -h 127.0.0.1 -p 8080
I tried running Deep Floor Plan with Python 3.6.10.
I tried running alembic, a Python migration tool
I took a quick look at the fractions package that handles Python built-in fractions.
I tried using the Python library "pykakasi" that can convert kanji to romaji.
I tried running two Jetson Nano hardware PWMs from the Jetson.GPIO Python library.
I tried to graph the packages installed in Python
A story that was convenient when I tried using the python ip address module
I tried running pymc
I tried Python on Mac for the first time.
I tried running the app on the IoT platform "Rimotte"
I tried to touch the CSV file with Python
I tried Python> autopep8
I tried running python etc. from a bat file
I tried to solve the soma cube with python
I tried python on heroku for the first time
mong --I tried porting the code that randomly generates Docker container names to Python -
[Python] I tried to graph the top 10 eyeshadow rankings
[Python] I tried running a local server using flask
A memo that I touched the Datastore with python
I tried to solve the problem with Python Vol.1
I felt that I ported the Python code to C ++ 98.
AWS Lambda now supports Python so I tried it
I tried Python> decorator
I tried running TensorFlow
I tried hitting the API with echonest's python client
I tried running the sample code of the Ansible module
I tried to summarize the string operations of Python
I tried running the offline speech recognition system Julius with python in the Docker virtual environment
I tried to find the entropy of the image with python
I tried "gamma correction" of the image with Python + OpenCV
I tried to simulate how the infection spreads with Python
I checked the Python package pre-installed in Google Cloud Dataflow
I tried the accuracy of three Stirling's approximations in python
I tried using the Python library from Ruby with PyCall
I tried running PIFuHD on Windows for the time being
I tried running Movidius NCS with python of Raspberry Pi3
Creating a Python script that supports the e-Stat API (ver.2)
I tried "a program that removes duplicate statements in Python"
I tried face recognition from the video (OpenCV: python version)
I tried programming the chi-square test in Python and Java.
[Python] I tried to visualize the follow relationship of Twitter
I tried to implement the mail sending function in Python
[Python] I tried collecting data using the API of wikipedia