[https proxy with squid] How to represent https whitelist with url_regex

Introduction

I tried to create a whitelist for use with the https proxy that intercepts ssl. The environment to use is the following that was built last time. ** Create Proxy with Active Directory linkage and SSL interception with squid easily with docker **

In short, what should I do?

The URL of the whitelist or blacklist is expressed by url_regex as follows.

** When representing a specific domain, including subdomains ** The following example represents a domain called qiita.com that contains subdomains.

whitelist


^(https*://)*([^/][^/]*\.)*qiita\.com(:443|:80)*(/.*)*$

** When representing a specific domain, including subdomains. So, if you want to combine multiple domain strings that have something in common ** The following example represents a domain called slack.com or slack-edge, com that contains subdomains.

whitelist


^(https*://)*([^/][^/]*\.)*slack(-edge)*\.com(:443|:80)*(/.*)*$

Prepare the above whitelist as a text file and load it as url_regex at squid.com.

squid.com


acl whitelist  url_regex -i "/etc/squid/whitelist"
http_access allow whitelist

The process of finding a way to write

Example: for qiita.com

Whitelisting only fqdn will connect you to a malicious site.

For example, suppose you write this on your whitelist:

whitelist


qiita\.com

But this would match if the full path or subdomain had the same string.

match_url


https://example.com/qiita.com/exploit.js
https://qiita.com.example.com/exploit.js

So why not write this, using the continuity from the first protocol to allow a particular domain name, including subdomains?

whitelist


^https*://([^/][^/]*\.)*qiita\.com/

This was no good. Perhaps to get a certificate, I'm trying to connect to qiita.com: 443 first.

squid_log


 TCP_DENIED/407 4054 CONNECT qiita.com:443 - HIER_NONE/- text/html
 TCP_DENIED/407 4424 CONNECT qiita.com:443 - HIER_NONE/- text/html
 TCP_DENIED/200 0 CONNECT qiita.com:443 PROSPER2\\USERNAME HIER_NONE/- -

Let's consider the protocol and port number in the previous rule. Anyway, if it's HTTP, let's communicate even if you enter: 80. The beginning of the string is matched with or without the protocol notation, and the end of the FQDN is matched with or without the port number.

whitelist


^(https*://)*([^/][^/]*\.)*qiita\.com(:443|:80)*/

It was still useless. It hasn't changed. It ends with a port number, so the last slash would get in the way and not match.

squid_log


 TCP_DENIED/407 4054 CONNECT qiita.com:443 - HIER_NONE/- text/html
 TCP_DENIED/407 4424 CONNECT qiita.com:443 - HIER_NONE/- text/html
 TCP_DENIED/200 0 CONNECT qiita.com:443 PROSPER2\\USERNAME HIER_NONE/- -

In addition to the previous rule, there is a slash, or there is no slash, and the port number or FQDN is the end of the string. And said.

whitelist


^(https*://)*([^/][^/]*\.)*qiita\.com(:443|:80)*(/.*)*$

The root site was able to confirm communication at both https://qiita.com/ and the subdomain site was at https://zine.qiita.com/. Maybe this is OK.

Example: for slack.com

Let's allow slack sites as well.

whitelist


^(https*://)*([^/][^/]*\.)*slack\.com(:443|:80)*(/.*)*$

Looking at it in a browser, it's halfway. Screenshot from Gyazo

Looking at the log, I'm going to communicate with slack-edge.com.

squid_log


 TCP_DENIED/407 4082 CONNECT a.slack-edge.com:443 - HIER_NONE/- text/html
 TCP_DENIED/407 4452 CONNECT a.slack-edge.com:443 - HIER_NONE/- text/html
 TCP_DENIED/407 4082 CONNECT a.slack-edge.com:443 - HIER_NONE/- text/html
 TCP_DENIED/200 0 CONNECT a.slack-edge.com:443 PROSPER2\\USERNAME HIER_NONE/- -

So it looks like you could just put the string -edge ** or ** not ** after the string slack.

whitelist


^(https*://)*([^/][^/]*\.)*slack(-edge)*\.com(:443|:80)*(/.*)*$

Let's try it again in the browser. Screenshot from Gyazo Yeah, this looks okay.

at the end

** It's much easier to use dstdomain for whitelists and blacklists ** However, I also tried to study regular expressions by trial and error.

Recommended Posts

[https proxy with squid] How to represent https whitelist with url_regex
How to make an HTTPS server with Go / Gin
How to update with SQLAlchemy?
How to cast with Theano
How to Alter with SQLAlchemy?
How to separate strings with','
How to RDP with Fedora31
How to Delete with SQLAlchemy?
How to cancel RT with tweepy
Python: How to use async with
How to use virtualenv with PowerShell
How to deal with imbalanced data
How to install python-pip with ubuntu20.04LTS
How to deal with imbalanced data
How to get started with Scrapy
How to get started with Python
How to deal with DistributionNotFound errors
How to get started with Django
How to Data Augmentation with PyTorch
How to use FTP with Python
How to calculate date with python
How to install mysql-connector with pip3
How to INNER JOIN with SQLAlchemy
How to install Anaconda with pyenv
How to authenticate with Django Part 2
How to authenticate with Django Part 3
Set up reverse proxy to https server with CentOS Linux 8 + Apache mod_ssl
[https proxy with squid] Avoid reading slack messages because it was websocket.
How to do arithmetic with Django template
[Blender] How to set shape_key with script
How to title multiple figures with matplotlib
How to get parent id with sqlalchemy
How to add a package with PyCharm
How to install DLIB with 2020 / CUDA enabled
How to use ManyToManyField with Django's Admin
How to use Cmder with PyCharm (Windows)
How to prevent package updates with apt
How to work with BigQuery in Python
How to use Ass / Alembic with HtoA
How to deal with enum compatibility errors
How to use Japanese with NLTK plot
How to do portmanteau test with python
How to search Google Drive with Google Colaboratory
How to display python Japanese with lolipop
How to download youtube videos with youtube-dl
How to power off Linux with Ultra96-V2
How to scrape websites created with SPA
How to use CUT command (with sample)
How to enter Japanese with Python curses
How to install zsh (with .zshrc customization)
How to read problem data with paiza
How to use SQLAlchemy / Connect with aiomysql
How to get started with laravel (Linux)
How to group volumes together with LVM
How to install python3 with docker centos
How to use JDBC driver with Redash
[https proxy with squid] Resolution of error caused by dh key too small
How to selectively delete past tweets with Tweepy
How to upload with Heroku, Flask, Python, Git (4)
How to deal with memory leaks in matplotlib.pyplot
How to create sample CSV data with hypothesis