ipsec: Don't match on packet mark for FWD XFRM policy

While extending datapath coverage, Martynas found a new bug affecting
IPsec + VXLAN + endpoint routes. In that configuration, cross-node
pod connectivity seems to fail and we see the IPsec XfrmInNoPols error
counter increasing.

By tracing the connection, we can see that the packet disappears on the
receiving node, after decryption, between bpf_overlay (second traversal)
and the lxc device. On that path, given decryption already happened,
we should match the FWD XFRM policy:

    src 0.0.0.0/0 dst 10.244.0.0/24 uid 0
	dir fwd action allow index 106 priority 2975 share any flag  (0x00000000)
	lifetime config:
	  limit: soft (INF)(bytes), hard (INF)(bytes)
	  limit: soft (INF)(packets), hard (INF)(packets)
	  expire add: soft 0(sec), hard 0(sec)
	  expire use: soft 0(sec), hard 0(sec)
	lifetime current:
	  0(bytes), 0(packets)
	  add 2023-01-13 14:34:18 use 2023-01-13 14:34:22
	mark 0/0xf00

Clearly, given the non-zero XfrmInNoPols, the packet doesn't match the
policy. Note XfrmInNoPols is also reported for FWD; there is no
XfrmFwdNoPols.

Checking the source code [1], we can see that when endpoint routes are
enabled, we encode the source security identity into the packet mark.
We thus won't match the 0/0xf00 mark on the FWD XFRM policy.

We shouldn't need to match on any packet mark for the FWD XFRM policy;
we want to allow all packets through. This commit therefore removes the
packet mark match for the FWD direction.

1 - https://github.com/cilium/cilium/blob/v1.13.0-rc4/bpf/lib/l3.h#L151-L154
Signed-off-by: Paul Chaignon <[email protected]>

pkg/datapath/linux/ipsec/ipsec_linux.go

@@ -196,11 +196,11 @@ func _ipSecReplacePolicyInFwd(src, dst *net.IPNet, tmplSrc, tmplDst net.IP, prox
 	policy := ipSecNewPolicy()
 	policy.Dir = dir
 	policy.Dst = dst
-	policy.Mark = &netlink.XfrmMark{
-		Mask: linux_defaults.IPsecMarkMaskIn,
-	}
 	if dir == netlink.XFRM_DIR_IN {
 		policy.Src = src
+		policy.Mark = &netlink.XfrmMark{
+			Mask: linux_defaults.IPsecMarkMaskIn,
+		}
 		if proxyMark {
 			// We require a policy to match on packets going to the proxy which are
 			// therefore carrying the proxy mark. We however don't need a policy