problem

Docker Hub is a free account and unauthenticated user, Pull number will be limited from November. If you operate on CircleCI without authentication, IP address-based restrictions will be applied, and CircleCI's IP addresses are not many compared to users, so there are concerns about the impact.

The CircleCI team seems to be talking to Docker, so it may not be necessary for users to respond in the future. However, if the CI / CD environment stops, the impact will be great, so let's hold down the workaround.

solution

If you bite the user authentication of Docker Hub for the time being, it will not be judged by the IP address, so you can use up to 200 pulls / 6h even with a free account. If it becomes more than this, you need to charge Docker Hub, but in any case, you need to authenticate the user on CircleCI, so the same work will occur.

Work procedure

Follow the steps below to authenticate Docker Hub users on CircleCI. Click here for the entire config.yaml.

https://github.com/AlisProject/serverless-application/blob/master/.circleci/config.yml -Click here for this fix

Set Docker Hub username and password in CircleCI Context

The procedure for registering in Context will come out soon, so I will omit it.

Add Context to Job that uses Docker

It seems that multiple Contexts are supported from 2020/09/15! This is nice. ..

  staging_steps: &staging_steps
    <<: *only_master
    context:
      - serverless_staging
      - docker-hub-credencials
  production_steps: &production_steps
    <<: *only_master
    context:
      - serverless_production
      - docker-hub-credencials

Add authentication to the part using Docker

Docker executor and the authentication information set in the place where Docker is pulled in Machine executor will be added. Below is an example of a Docker executor.

Create an item for reference

references:
  docker_hub_authentication: &docker_hub_authentication
    auth:
      username: $DOCKERHUB_USER
      password: $DOCKERHUB_PASSWORD

Applies to all areas where the image is pulled

The following is an example of applying to the parts summarized in ʻexecutors`.

executors:
  python3_6_1:
    working_directory: ~/repo
    docker:
      - image: circleci/python:3.6.1
        <<: *docker_hub_authentication
  test:
    working_directory: ~/repo
    docker:
      - image: circleci/python:3.6.1
        <<: *docker_hub_authentication
      - image: bluszcz/bflocalstack-dynamodb-s3
        <<: *docker_hub_authentication
      - image: alismedia/dynamodb-local
        <<: *docker_hub_authentication
        environment:
          MAX_HEAP_SIZE: 4096m
          HEAP_NEWSIZE: 3072m
      #No credentials required as it is not Docker Hub
      - image: docker.elastic.co/elasticsearch/elasticsearch:6.2.0
        environment:
          discovery.type: single-node

If you can make the above settings and build the same as before the change, you are done.

error

If you accidentally add your credentials to a location other than Docker Hub, you will get the following error:

Error response from daemon: Get https://docker.elastic.co/v2/elasticsearch/elasticsearch/manifests/6.2.0: unauthorized: authentication required

The following is a mysterious error. This is the first time I saw it.

Allocating a remote Docker Engine
Requesting version: 18.09.3

Got error while creating host: failed to create host: rpc error: code = Unknown desc = failed to create VM: Put "https://vm-service.infra.circleci.com/tasks/5f7ffcbaa7b0aa6a8ec29c79-0-build%2F59C7C9D1": EOF
We had an unexpected error preparing a VM for this build, potentially due to our infrastructure or cloud provider.  Please retry the build in a few minutes

Rerun as instructed and it was resolved.

Settings to bypass Docker Hub restrictions on CircleCI