Re-examined Linux PBR

What is PBR?

Often written as Policy Base Routing or Policy-Based Routing. (Is the latter a Cisco system?) In this article, I will write about Linux PBR, but if you want to know about Cisco PBR, please refer to here.

When setting up routing in Linux, the routing table is used, but in fact, there are multiple routing tables and there is a routing policy that decides which routing table to use. PBR is performed by utilizing this routing policy.

Basics of PBR

This is beautifully organized so that it feels good. [How Linux chooses a packet route](How Linux chooses a packet route)

First, let's take a look at the default routing policy.

# ip rule
0:      from all lookup local
32766:  from all lookup main
32767:  from all lookup default

In this way, in the initial state, it is set to go to local, main, default in that order. It is not set for 1-32765, but you can add the policy manually. How the routing policy refers to the routing table is

1. Browse cache

2. Refer to the local table if it is not in the cache
3. If it is not in the local table, refer to the manually added table (if any)
4. Refer to the main table if it is not in the manually added table
5. If it is not in the main table, refer to the default table

In this way, if there is no table written in the next rule, if not, it goes to the routing table written in the next rule. And the contents of the routing table are referenced like this.

1. Select the most specific (long netmask) route among the matching routes

2. If there are multiple routes with the same netmask length, select the matching route with TOS set.
3. If there are multiple routes with the same TOS, select the route with the smaller metric.
4. If there are multiple routes with the same metric, select the last added route.

Matching route means that the destination IP (range) written in the routing table matches the destination IP of the packet you want to send. In the routing table, the destination IP is used as the reference, so if you want to route using other criteria, you need to use a routing policy earlier than this.

Source policy routing

Roughly speaking, "I want to select a routing table based on the source IP".

See this area for details. Linux Advanced Routing & Traffic Control HOWTO Routing settings for Linux machines with two NICs policy-based ip routing management Pointing to two NICs to the same subnet, or story of source routing Linux Policy Base Routing

Now let's look at the default rules again.

# ip rule
0:      from all lookup local
32766:  from all lookup main
32767:  from all lookup default

It says from all, which matches all outbound IPs. Let's add a rule that specifies the source.

#ip rule add from 172.10.10.1/24 table 100
#ip rule
0:      from all lookup local
32765:  from 172.10.10.1/24 lookup 100
32766:  from all lookup main
32767:  from all lookup default

I'm afraid if I can't log in by mistakenly specifying the source of the route actually used for SSH etc., so I will limit the destination as well. You can also speed up the reference order by specifying a younger number with prio.

#ip rule add from 172.10.10.2/24 to 172.20.20.1/32 table 101 prio 100
#ip rule
0:      from all lookup local
100:    from 172.10.10.2/24 to 172.20.20.1 lookup 101
32765:  from 172.10.10.1/24 lookup 100
32766:  from all lookup main
32767:  from all lookup default

If the source IP is not specified

Especially if the application does not specify the source IP (for example, when you usually hit Ping, the command includes only the destination IP, not the source IP. Of course, specify it with the -I option. It is possible, though.) Apparently, it matches the from all rule. Then, a matching route is found and the source IP is determined, but it seems that the routing policy is referenced again with the information of this source IP.

Since the source IP has been decided this time, there is a possibility of hitting not only from all but also from XX.XX.XX.XX.

The reason why the routing policy is referenced twice is that if you look at how the packet is flowing inside Linux, it is because the packet has passed the Routing Decision twice after leaving the Local Process. I think it will be.

If the source IF is not specified

Especially if the application does not specify a source IF, it will likely follow the'dev XXXX' notation for matched routing.

#ip route
default via 172.19.48.1 dev eth0
172.19.48.0/20 dev eth0 proto kernel scope link src 172.19.49.170

In other words, even if it is a response to a packet coming from outside, it does not always return a response from the received IF, and it adheres to the routing. (For the source IP, the IP when it was received should be used)

Determining the sender

I didn't know what it was, so I verified it. The tools used for validation are ping and traceroute. As mentioned earlier, ping allows you to specify the source with the -I option. Also, traceroute allows you to specify the source with the -i and -s options.

In addition, we will call it as follows here.

--Enter the dev XXXX part described in the routing IF --The part of src X.X.X.X described in the routing is described IP --There is a specification that is actively set by the application --Nothing specified in the application that was not set

result

I want to have a finer routing policy

By marking using a Linux firewall, you can write a routing policy for that marking.

Please see here for details. Apply DS-Lite only to ports 80 and 443 using FWMARK Iptables Tutorial 1.2.2

The reason for doing this is that the patterns that can be specified in the Linux firewall (for example, using the iptables command) are very detailed, and this gives you more flexibility in writing rules that match packets.

However, there are some caveats.

From Complete control of traffic with NETMARK and iproute2

[Note] When controlling routing using fwmark with iproute2, it seems that if the default gateway is not set, it will be treated as "Network Unreachable" before going to the fwmark table, so be careful.

Please be aware that you may have been addicted to this once. (At this time, it was CentOS 7.3) Probably, it goes through the Routing Decision immediately after leaving the Local Process, so if there is no matching routing here, it will be treated as "Network Unreachable".

I want to separate the routing for each process

It cannot be achieved with PBR, but it can be achieved by using NameSpace.

See this area for details. Separate Linux routing table using Network Namespace How to use the ip netns command (the range of network experiments will expand) ip-netns(8) — Linux manual page Trying to handle VXLAN on Linux

Let's use iproute2

So far, we've used the ip command to look at the contents of PBR, but I'd like to mention other methods as well. It is recommended that you limit the method used to change the settings to one as much as possible, or clarify the division of roles. (Otherwise, you will always batting somewhere and unexpected inconvenience will occur)

iproute2 This package contains the ip command and so on. In routing, we mainly use ip rule which can display and operate the routing policy and ip route which can display and operate the routing table. The contents set by this command will be reflected immediately, but they will disappear when you restart. In case of emergency, you can restart it, so you can take on the challenge with confidence. Basically, you should be able to use any distribution (even embedded system), so it is convenient to master iproute2.

Please see here for details. Linux Advanced Routing & Traffic Control HOWTO ip-rule(8) — Linux manual page ip-route(8) — Linux manual page Configure static routes with IP commands

network-tools A super-typical network-related package that includes ifconfig, arp, route, and netstat. However, it is deprecated today. Many people find it easier to use, but I recommend using iproute2 from now on.

NetworkManager

In RedHat system, network setting is often done with Netowrk Manager. There is a GUI, but TUI and CLI are also available, which can be handled by the nmtui command and nmcli command, respectively. For most network settings, nmtui is sufficient, but for more advanced settings like this one, use the CLI nmcli command. I'm not going to deal with nmcli settings in this article (for now), but this is more convenient if you normally set up your network with NetworkManager. However, PBR is not good enough and difficult to handle. First of all, I think it is better to get used to the operation with the ip command before using it.

Please see here for details. Configuring Policy-Based Routing to Define Alternate Routes (https://access.redhat.com/documentation/ja-jp/red_hat_enterprise_linux/8/html/configuring_and_managing_networking/configuring-policy-based-routing-to-define- alternative-routes_configuring-and-managing-networking) RHEL7 network settings

Write directly to the config file

Here, I will talk about Red Hat. (Sorry, I'm not familiar with debian ...) Netowork Manager, which I introduced earlier, has many environments that are intentionally stopped. If you don't want to use NetoworkManager but want the settings to remain even after a reboot, you can do so by writing directly to the config file. Again, I think it's best to get used to it with the ip command.

Please see here for details. I tried to set Policy Based Routing in RHEL 7.4. Setting static routes in IFCFG files Set up Policy Based Routing (PBR) on RHEL 7 and CentOS 7

Finally

Please forgive me though it is a cut-out article with only quotations and links.