Create a private repository with AWS CodeArtifact

June 2020 AWS Code Brothers suddenly reborn. Its name is AWS CodeArtifact. A private repository service that allows you to securely store, publish, and share software packages. It is already generally available in the Tokyo region and is also compatible with VPC Endpoint.

:rocket: Introducing AWS CodeArtifact: A fully managed software artifact repository service https://aws.amazon.com/jp/about-aws/whats-new/2020/06/introducing-aws-codeartifact-a-fully-managed-software-artifact-repository-service/

Similar tools and services include JFrog Artifactory and GitHub Package Registry, I think Azure Artifacts is famous.

What are you happy about

There are two main benefits to repository managers like AWS CodeArtifact.

1. Official repository cache

When developing software, create an ecosystem such as package managers for various languages It has become commonplace to use it.

CodeArtifact can be used as a proxy for corresponding package managers such as maven and npm. Administrators can control access to packages and are approved for use within the organization Only the latest packages will be available to developers.

Also, depending on your organization's policies, you may not be able to freely access public repositories. By using VPC Endpoint, it was cached in CodeArtifact without going through the Internet. You can access the package. You can also expect a reduction in download time.

As of June 2020, the following external connections are supported. It seems to be the minimum compared to other companies' services, but I look forward to future updates.

Type Description
npm npm public registry
Python Python Package Index
Maven Maven Central
Maven Google Android repository
Maven Gradle plugins repository
Maven CommonsWare Android repository

2. Manage your own package

CodeArtifact provides a central place to store packages built within your organization. Because the stored packages can be safely shared and made available to existing package managers This will reduce development time.

Important concepts to remember in CodeArtifact

Repository

You can create multiple repositories within your account. (Maximum 100) A repository can contain packages in multiple supported languages. The repository is the unit of endpoint when accessed from each package manager.

domain

CodeArtifact is a unique concept. Manage each repository in a single domain.

Each package is used via the repository, but the actual data is not the repository It is saved in the domain. Specific packages in multiple repositories within the same domain Even if you use it, it will be saved only once, regardless of the number of repositories. In other words, storage charges will not be duplicated.

Domains are also available for cross-accounts. Create a common domain for your organization Each account can use its own repository. image.png

Upstream repository

You can set other repositories as upstream repositories for each repository. This makes it a common repository within the organization, a repository for development teams, etc., depending on the application. Clients can still access from a single endpoint, even if they are separated. image.png

Domain policy and repository policy

It can be set in the familiar json format.

Domain policy settings for accounts within your organization or specific IAM principals You can allow access to the domain.

Similarly, if you define a resource policy, for users who can access the repository and for the repository You can specify the actions that can be performed. As a result, in the case of the above configuration example, only read permission from within the organization is set in the organization shared repository. The development team repository can be configured to allow only development members to read and write.

Try

Creating domains and repositories

Select Create Repository at the top of the CodeArtifact console. image.png Enter the repository name and in the public upstream repository settings Select the official repository to cache. image.png Then select the domain where you want to register the repository. This account has not created a domain yet, so specify a domain name and create a new one. Additional settings allow you to choose the KMS key to use for encryption, either managed or specific CMK. image.png Finally, check the settings and create the repository and domain. image.png As you can see from the package flow, in the repository creation wizard Only public upstream repositories are available for selection. To set another repository as an upstream repository, edit after creating the repository Must be set. image.png

How to connect to the repository

AWS CLI v2.0.21 or higher is required to get the connection information of the created repository. You can check the specific connection command for each tool from the repository screen of the console. image.png For example, in the case of pip: The --domain-owner option is optional for the account that created the domain.

$ aws codeartifact login --tool pip --repository my-repo --domain my-domain --domain-owner 0123456789012
Successfully logged in to codeartifact for pip.

Internally, it seems that pip config set automatically sets global.index-url. You can see the repository endpoints with credentials in the pip config list. This credential expires after 12 hours and must be done each time.

Alternatively, use the aws codeartifact get-authorization-token command to get only the authentication token. You can also do a manual pip config set.

It seems that the logout command is not prepared, so to cancel it There seems to be no choice but to run pip config unset global.index-url.

Public repository cache

If you try installing the AWS SAM CLI, you will see that it is downloaded from CodeArtifact.

$ pip install -U aws-sam-cli
Collecting aws-sam-cli
  Downloading https://my-domain-123456789012.d.codeartifact.ap-northeast-1.amazonaws.com/pypi/my-repo/simple/aws-sam-cli/0.52.0/aws_sam_cli-0.52.0-py3-none-any.whl (443kB)

You can also see that it is registered in the created repository on the console. In fact, the public upstream repository, pypi-store, connects to and caches PyPI. image.png

Sharing your own package

Here is provided for the Python Packaging User Guide tutorial Try uploading the sample project to CodeArtifact.

Clone https://github.com/pypa/sampleproject.

$ git clone https://github.com/pypa/sampleproject.git
Cloning into 'sampleproject'...
remote: Enumerating objects: 21, done.
remote: Counting objects: 100% (21/21), done.
remote: Compressing objects: 100% (19/19), done.
remote: Total 441 (delta 7), reused 6 (delta 1), pack-reused 420
Receiving objects: 100% (441/441), 116.64 KiB | 365.00 KiB/s, done.
Resolving deltas: 100% (216/216), done.

Package with the following command.

$ cd sampleproject
$ python setup.py bdist_wheel

For Python, twine is required to upload the package, so install it.

$ pip install -U twine

Set the twine credentials with the login command. Internally, the settings for CodeArtifact connection are added to ~ / .pypirc.

$ aws codeartifact login --tool twine --repository my-repo --domain my-domain
Successfully logged in to codeartifact for twine.

Upload to CodeArtifact with the following command.

$ twine upload -r codeartifact dist/*
Uploading distributions to https://my-domain-123456789012.d.codeartifact.ap-northeast-1.amazonaws.com/pypi/my-repo/
Uploading sampleproject-1.3.1-py3-none-any.whl

I was able to confirm that it was uploaded on the console as well. You can easily share your own package. image.png

reference

** AWS CodeArtifact product page ** https://aws.amazon.com/jp/codeartifact/ CodeArtifact User Guide https://docs.aws.amazon.com/codeartifact/latest/ug/welcome.html ** Publish / upload a library created in Python to PyPI ** https://qiita.com/icoxfog417/items/edba14600323df6bf5e0

that's all. I'm glad if you can use it as a reference.

Recommended Posts

Create a private repository with AWS CodeArtifact
Create a local pypi repository
Create a Layer for AWS Lambda Python with Docker
Create a heatmap with pyqtgraph
Create a directory with python
[AWS] Create API with API Gateway + Lambda
Create a virtual environment with Python!
Create a poisson stepper with numpy.random
Create a file uploader with Django
Deliver private content with AWS CloudFront
[AWS Hands-on] Let's create a celebrity identification service with a serverless architecture!
Create a private DMP with zero initial cost and zero development with BigQuery
How to create a serverless machine learning API with AWS Lambda
Create a Python function decorator with Class
Build a blockchain with Python ① Create a class
Create a dummy image with Python + PIL.
[Python] Create a virtual environment with Anaconda
Let's create a free group with Python
Create a GUI app with Python's Tkinter
Issue a signed URL with AWS SQS
Create a large text file with shellscript
Create a VM with a YAML file (KVM)
Create a word frequency counter with Python 3.4
How to create a repository from media
Create a Connecting Nearest Neighbor with NetworkX
Create a web service with Docker + Flask
Create a car meter with raspberry pi
Create a devilish picture with Blender scripts
Create a matrix with PythonGUI (text box)
Create a graph with borders removed with matplotlib
[AWS] Create a Python Lambda environment with CodeStar and do Hello World
Let's create a chat function with Vue.js + AWS Lambda + dynamo DB [AWS settings]
Create a frame with transparent background with tkinter [Python]
[AWS SAM] Create API with DynamoDB + Lambda + API Gateway
Build a WardPress environment on AWS with pulumi
Create a bot with AWS Lambda that automatically starts / stops Instances with specific tags
Try Tensorflow with a GPU instance on AWS
Create a GUI executable file created with tkinter
Create a LINE BOT with Minette for Python
Create a game UI from scratch with pygame2!
Create a PDF file with a random page size
Create a virtual environment with conda in Python
Create a page that loads infinitely with python
[Note] Create a one-line timezone class with python
You can easily create a GUI with Python
Create a python3 build environment with Sublime Text3
Pipenv install with ssh from Private Bitbucket Repository
Build a cheap summarization system with AWS components
Create a bulletin board with Heroku, Flask, SQLAlchemy
Create a dashboard for Network devices with Django!
Create a matrix with PythonGUI (tkinter combo box)
Create a color bar with Python + Qt (PySide)
Steps to create a Twitter bot with python
Create a decision tree from 0 with Python (1. Overview)
Create a new page in confluence with Python
Create a color-specified widget with Python + Qt (PySide)
How to create a multi-platform app with kivy
Create a one-file hello world application with django
AWS Step Functions to learn with a sample
Create a Photoshop format file (.psd) with python
Create a Python console application easily with Click