Simple "IPsec" implementation

[p4pe]

Hello (again) I'm currently trying to build a CPU intensive VNF so Ι decided to creat simple IPsec-alike flow. I just want to implement the flow for the packets that entering the tunnel.

My topology is the same that I used here, and I just change the middle router in order to act as an IPsec, that pull the traffic from the FastUDPsource element.
I show the example and for my configuration I'm using:
FromDevice(eth0)->IPsecESPEncap()->IPsecAuthHMACSHA1(0)->IPsecAES(1)->IPsecEncap(50)->Queue->ToDevice(eth1)

Probably this is not right because I had the following error "Segmentation fault (core dumped)

Kindly thank you (again) for your input.

[ahenning]

The ipsec config does not look complete. I believe the missing part is the RadixIPsecLookup, which if I recall correctly sets annotations required by the other ipsec elements.

[ahenning]

For what it is worth, here is a simple example to get started with

InfiniteSource(LENGTH 1000, LIMIT 3, STOP true)
-> SetRandIPAddress(192.168.0.0/24, 32)
-> UDPIPEncap(1.1.1.1,123, DST_ANNO, 123)
-> ipsec_rt::RadixIPsecLookup( 10.10.1.1/32 0,
192.168.0.0/24 10.10.2.1 1 111 1111111111111111 1111111111111111 1 0 ,
0.0.0.0/0 2
);

ipsec_rt[0]
// Receive
-> Discard;
ipsec_rt[1]
-> IPsecESPEncap()
-> IPsecAuthHMACSHA1(0)
-> IPsecAES(1)
-> UDPIPEncap(10.10.1.1 , 4500 , DST_ANNO , 4500)
-> IPPrint("IPsec packet")
-> Discard;
ipsec_rt[2]
// Default
-> Discard;

[p4pe]

Thank you for you answer @ahenning. I saw the RadixIPsecLookup and I was wondering if it is mandatory for the implementation.

Thank you for the example. I will try it.

[p4pe]

Hello @ahenning I run this configuration thank you. For my scenario my scenario I will generate the traffic to source using the InfiniteSource element.

I'm thinking a click configuration file like this:

InfiniteSource(LENGTH 1000, LIMIT 3, STOP true)
-> SetRandIPAddress(192.168.0.0/24, 32)
-> ToDevice(ethX)

And the IPSec VNF will be like this:
FromDevice(ethX) -> RadixIPSecLookup(..)

Correct me if I'm wrong.

I also try to create a simple NAT configuration file, kindly ask you for some input.
Here is my first attempt.
rw::IPRewriter(pattern 27.32.11.3 1000 - - 0 1);

InfiniteSource(LENGTH 1500, LIMIT 10000, STOP true)
-> UDPIPEncap(192.168.1.6, 50 192.168.1.13, 100, 3)
-> IPPrint(sendPacketsToDst)
-> [0]rw;

rw[0] -> IPPrint(NAT) -> IPMirror -> IPPrint(backToSource)-> [0]rw;
rw[1] -> IPPrint(rw1) -> Discard;

Thank you again

[p4pe]

Hello again @ahenning I am kindly asking for your input on something.

As I told before I have a simple implemation with three nodes Source --> VNF --> Sink
I tried two different implementations for my scenario.
First, in the source I have a click configuration file
src :: FastUDPSource(1,-1,60,3c:fd:fe:04:64:42,192.168.2.2,1099, 3c:fd:fe:04:7b:a2, 192.68.2.5,1100);
td::ToDevice(enp4s0f1);
src -> Counter -> td

And the second again in the source
InfiniteSource(LENGTH 1000, LIMIT 3, STOP true)
-> SetRandIPAddress(192.168.0.0/24, 32)
-> UDPIPEncap(192.168.2.2,1234, DST_ANNO, 123)
-> IPPrint("UDP Packet send to destination")
-> ToDevice(enp4s0f1)

On the VNF node I have the IPSec implementation
****fd:: FromDevice(enp4s0f1);
rt:: RadixIPsecLookup(192.168.1.2/32 0,
192.168.2.5/24 10.10.2.1 1 111 1111111111111111 1111111111111111 1 0,
0.0.0.0/0 2 );

fd -> rt

rt[0]
// Receive
-> Discard;
rt[1]
-> IPsecESPEncap()
-> IPsecAuthHMACSHA1(0)
-> IPsecAES(1)
-> UDPIPEncap(10.10.1.1 , 4500 , DST_ANNO , 4500)
-> IPPrint("IPsec packet")
-> Queue
-> Counter
-> ToDevice(enp4s0f0);
rt[2]
// Default
-> Discard;****

Using tcpdump on the enp4s0f1 interface on VNF, I realize that tcpdump did not "catch" any packet when I use InfiniteSource as packet generator, and obviously the IPSec flow did not work.

Using the FastUDP source, tcpdump catches packets, but it seems my IPSec configuration did not work.

Kindly ask you for some input.

Thank you in advance

[ahenning]

The issue appears to be the L2/MAC headers. In the first example it looks like this header is set, while in the second there are no MAC headers. EtherEncap might help.

By default FromDevice(ethX) would provide the MAC headers, so you would need to Strip(14) to remove it, CheckIPHeader to ensure its a valid packet, which should normally be done before using elements that expect IP packets. Then afterwards Unstrip(14) can be used to 'add' the L2 header back, e.g. before using elements expecting an ethernet packet e.g. ToDevice.

Just a rough off hand example:
FromDevice(eth0) -> Strip(14) -> CheckIPHeader -> /* IP elements */ -> Unstrip(14) -> ToDevice(eth1);

The L2 headers at the source is one issue, there might be more, for example in the IPsec config after FromDevice and before ToDevice.

[p4pe]

Thank you @ahenning also I forgot to say that all the interfaces are in promisc mode.

[p4pe]

So your advice @ahenning is to configure the VNF node like this:

fd:: FromDevice(enp4s0f1);
fd -> Strip(14) ->CheckIPHeader -> rt:: RadixIPsecLookup() -> Unstrip(14) ->ToDevice()

[ahenning]

Yes and no. Yes that is how L2 headers could work if the VNF was just a L2 bridge, but due to the udp and ipsec headers you are probably going to be better off using EtherEncap after UDPIPEncap and before ToDevice

[p4pe]

It seems that I am close enough to my target, but I am little bit confuse.

Finally I have the following IPSec config file:

in::FromDevice(enp4s0f0)
  -> IPPrint("I took the packet")
  -> Strip(14)
  -> CheckIPHeader()
  -> GetIPAddress(16)
  -> ipsec_rt::RadixIPsecLookup( 10.10.1.1/32 0,
192.168.2.0/24 10.10.2.1 1 111 1111111111111111 1111111111111111 1 0 ,
0.0.0.0/0 2
);




ipsec_rt[0]
// Receive
-> Discard;
ipsec_rt[1]
-> IPsecESPEncap()
-> IPsecAuthHMACSHA1(0)
-> IPsecAES(1)
-> UDPIPEncap(10.10.1.1 , 4500 , 192.168.2.5 , 4500)
-> EtherEncap(0x800,1:1:1:1:1:1,a0:36:9f:08:20:80)
-> IPPrint("this is an IPSec packet")
-> Queue
-> ToDevice(enp4s0f1);
ipsec_rt[2]
// Default
-> Discard;

The following configuration on the source node:

FastUDPSource(1, 10000000, 60, a0:36:9f:08:22:60, 192.168.2.2, 1234,           
                                     a0:36:9f:08:20:80, 192.168.2.5, 1234)
  -> IPPrint("Hello")                                                          
  -> ToDevice(enp4s0f0);

I can capture t the traffic in the ipsec node:

And I can capture the traffic with tcpdump on the interface enp4s0f1 that connects the IPSec node with the sink.

But on the sink node with the following configuration

FromDevice(enp4s0f1) -> c:: Counter -> Discard;
DriverManager(wait 45s, save c.rate -, stop)

I could not capture any traffic on the interface.

Do you see any error here @ahenning?

Thank you in advance.

[ahenning]

I notice 'enp4s0f1' in the ToDevice of the ipsec config and the FromDevice of the sink. If this is actually the exact same interface, that could be the issue. If not, then perhaps the FromDevice on the sink needs 'PROMISC true'. I don't see any major issues, looks like you are 99% there.

Also, I'd recommend to run tcpdump with flag '-e' to print the L2 headers. This might help to spot issues with the packets.

[p4pe]

No these are different interfaces on two different nodes. I have also enable promisc mode on this interface with the command

ip link set <interface> promisc on

what does it mean the indication expensive::Push have 0 wanted 14

[ahenning]

What is between the two enp4s0f1 interfaces? Might want to look if there is some L2 filter going on that might be filtering the spoofed source MAC (1:1:1:1:1:1), or just set it to the actual MAC of the interface.

If you can see the packets leaving the interface with tcpdump, and you are sure the packet headers are valid, then they should arrive on the destination interface.

[p4pe]

The topology is like this:

Source(Node1) (enp4s0f0) <-------->(enp4s0f0)(Node2)VNF(enp4s0f1) <-------> (enp4s0f0)(Node3)Sink

Source: 192.168.2.2/24, a0:36:9f:08:22:60
VNF: 192.168.2.3/24 , 192.168.2.4/24
Sink: 192.168.2.5/24, a0:36:9f:08:20:80

A silly question, in the EtherEncap() the source mac address should be the the mac address of the eggress interface (i.e. enp4s0f) ?

[ahenning]

Yes, some L2 devices, physical or virtual, will check if the packets are sourced from the connected interface MAC. I am not sure if this the issue, but double check the L2 header with tcpdump '-e'. If the packets are valid, the issue is probably not click related.

I would make one change to the sink config. Replace DriverManager with Script(wait 10, print c.rate, loop); or something along those lines.

[p4pe]

@ahenning It seems that the source mac address in the EtherEncap was the problem.. I fixed it and now I can capture the packets on the ingress interface of the sink..

Now I can count the rate with the Counter, but I can't explain why sometimes the rate is 0 and the other 23887, the fluctuation is weird

[p4pe]

Hello @ahenning, and happy new year.

I m facing an issue, and i was wondering if you can provide me some input.
Using the above IPSec implementation, I observe that I had a throughput issue.

I send 500k packet/sec with FastUDP, I measure the rate(using AverageCounter element) on the two VNF ports (ingress - egress) and on the sink, and I can't find where is the bottleneck.

Source (500023,13) -> (212191,40)VNF(210629,83) -> (214183)Sink.

I also noticed that after some interation I had a warning on the VNF: CheckIPHeader@3: IP header check failed: bad IP version

I use the drop_details handler of the CheckIPHeader element to see if this is the root of the problem. But the output show that only one packet is dropped every time

What could cause this throughput "issue", except from that I use Click instead of FastClick?

Thank you in advance

[ahenning]

Packets per second looks normal for the default PCAP method. You can try the fastclick DPDK elements for higher throughput.

You can also try enabling verbose logging on the IP header check element. Perhaps it is just promiscuous mode picking up something else on the network. You might be able to output the error packets to a second output port and print the packets to get more detail on the packet that causes the error.

...
-> check::CheckIPHeader(VERBOSE true);

check[0] -> out;
check[1] -> Print -> Discard;

[p4pe]

I believe that the packet loss is due to the NICs buffers, so yes the p/sec looks normal.
I was wondering, if i could run two instances of the IPSec, on the same CPU core.
I tried with click -a ipsec.click and click -a ipsec1.click, the two processes running but on different cores.

[ahenning]

review linux process pinning to pin process to a core

[p4pe]

I just said if there is any other way than the obvious one :)