Find the answer to your Linux question:
Results 1 to 4 of 4
I have two machines with Debian squeeze (kernel 2.6.32-5-amd64) connected via GRE tunnel on top of IPSec transport mode (openswan) with 3des encryption. Code: |m1|-eth0------------------|cloud|------------------eth0-|m2| | | |--gre0-----------------IPSec + GRE---------------gre0-| ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Oct 2010
    Posts
    7

    IPSec + GRE/openvswitch poor performance


    I have two machines with Debian squeeze (kernel 2.6.32-5-amd64) connected via GRE tunnel on top of IPSec transport mode (openswan) with 3des encryption.
    Code:
    |m1|-eth0------------------|cloud|------------------eth0-|m2|
       |                                                      |
       |--gre0-----------------IPSec + GRE---------------gre0-|
                             192.168.1.0/24
    Both eth0 interfaces are 100Mbit Ethernet.

    I have made some test with scp and iperf tools:
    1. Connection without IPSec via eth0
    Code:
    #time scp file m2:
    file                 100%  271MB  11.3MB/s   00:24    
    
    real    0m27.425s
    user    0m3.052s
    sys     0m0.392s
    
    # iperf -c m2 -p 6666
    ------------------------------------------------------------
    Client connecting to m2, TCP port 6666
    TCP window size: 16.0 KByte (default)
    ------------------------------------------------------------
    [  3] local m1 port 51003 connected with m2 port 6666
    [ ID] Interval       Transfer     Bandwidth
    [  3]  0.0-10.0 sec    112 MBytes  94.0 Mbits/sec
    2. Connection without IPSec via GRE tunnel
    Code:
    #time scp file m2:
    file                 100%  271MB  10.1MB/s   00:27    
    
    real    0m34.369s
    user    0m3.032s
    sys     0m0.460s
    
    # iperf -c m2 -p 6666
    ------------------------------------------------------------
    Client connecting to m2, TCP port 6666
    TCP window size: 16.0 KByte (default)
    ------------------------------------------------------------
    [  3] local m1 port 51003 connected with m2 port 6666
    [ ID] Interval       Transfer     Bandwidth
    [  3]  0.0-10.0 sec    109 MBytes  91.1 Mbits/sec
    3. Connection with IPSec via eth0
    Code:
    #time scp file m2:
    file                 100%  271MB  10.9MB/s   00:25    
    
    real    0m28.075s
    user    0m3.064s
    sys     0m1.952s
    
    # iperf -c m2 -p 6666
    ------------------------------------------------------------
    Client connecting to m2, TCP port 6666
    TCP window size: 16.0 KByte (default)
    ------------------------------------------------------------
    [  3] local m1 port 51003 connected with m2 port 6666
    [ ID] Interval       Transfer     Bandwidth
    [  3]  0.0-10.0 sec    109 MBytes  91.6 Mbits/sec
    4. Connection with IPSec via GRE tunnel
    Code:
    #time scp file m2:
    file                 9%   27MB 452.4KB/s   09:13 ETA^C
    
    real    1m1.899s
    user    0m0.352s
    sys     0m56.400s
    
    # iperf -c m2 -p 6666 -i 10 -t 60
    ------------------------------------------------------------
    Client connecting to m2, TCP port 6666
    TCP window size: 85.3 KByte (default)
    ------------------------------------------------------------
    [  3] local m1 port 37725 connected with m2 port 6666
    [ ID] Interval       Transfer     Bandwidth
    [  3]  0.0-10.0 sec  10.4 MBytes  8.71 Mbits/sec
    [  3] 10.0-20.0 sec  8.54 MBytes  7.16 Mbits/sec
    [  3] 20.0-30.0 sec  7.47 MBytes  6.27 Mbits/sec
    [  3] 30.0-40.0 sec  6.75 MBytes  5.66 Mbits/sec
    [  3] 40.0-50.0 sec  6.20 MBytes  5.20 Mbits/sec
    [  3] 50.0-60.0 sec  5.75 MBytes  4.82 Mbits/sec
    [  3]  0.0-60.0 sec  45.1 MBytes  6.30 Mbits/sec
    What is wrong with 4th test? Why the connection is so slow? Both tools uses 100% cpu time during only this trial. My servers have i7-920 (2.67GHz). I have also tried with openvswitch software which is able to send Ethernet over GRE. The results were the same. The last idea which came to my mind was openvpn. I set up openvpn in bridge mode also on top of IPSec. The results were only a little worse than the third trial. Any suggestions? Maybe it is a bug in linux kernel?

    best regards
    Maciej Galkiewicz
    Last edited by maciejgalkiewicz; 10-15-2010 at 12:57 PM.

  2. #2
    Linux Engineer Kloschüssel's Avatar
    Join Date
    Oct 2005
    Location
    Italy
    Posts
    773
    Looks to me as if IPSec over GRE is being done in software and not in hardware (IPSec over eth0 probably is done in hardware). Encryption can be really slow if it is not hardware accelerated.

    EDIT:

    in fact my assumptions seem to be right:

    A quote from NIL - Data Center Interconnections: Technical Implementations yields following:

    GRE over IPSec over the Internet
    [...]
    To achieve high performance, this technology requires special hardware, such as VPN acceleration modules to encrypt and decrypt traffic at near-wire speeds.
    Last edited by Kloschüssel; 10-18-2010 at 10:22 AM.

  3. #3
    Just Joined!
    Join Date
    Oct 2010
    Posts
    7
    Quote Originally Posted by Kloschüssel View Post
    Looks to me as if IPSec over GRE is being done in software and not in hardware (IPSec over eth0 probably is done in hardware). Encryption can be really slow if it is not hardware accelerated.
    This is GRE over IPSec not conversely. I don't think that there is hardware acceleration. My server has RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02). BTW the cpu i7-920 is really efficient.

  4. $spacer_open
    $spacer_close
  5. #4
    Linux Engineer Kloschüssel's Avatar
    Join Date
    Oct 2005
    Location
    Italy
    Posts
    773
    This is GRE over IPSec not conversely.
    My error, sorry.

    My server has RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 02). BTW the cpu i7-920 is really efficient.
    Maybe packet fragmentation is the evil. Try to decrease the MTU size in your tunnel by 68 bytes.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •