Efficient Multicore Networking

[yalakvar]

Hi,

I am working on a design for some kind of a firewall that needs to handle heavy traffic. In my application I have a few raw socket file descriptors that I need to receive from. It is important to me that to design my application to be efficient and scalable on multicore architecture.

In order to do so I thought about creating receiving threads as many as the number of cores. Each thread will call select() on ALL sockets FDs. I am not sure this is a safe design. I am also not sure what will happen in case only one socket will be contended, will it scale?

I guess I can use a single thread that calls select(), put the received packets in a queue, and let processing threads to process it. I do prefer the first design because I don't want to pay the queue and the context switch overheads.

Is the design using select() in multiple threads is a correct design?

Thank you for your help,
Mickey

[cipherus]

Some people would say that using multiple processes to utilize multi-core CPU's will actually slow down your network handling and also allow you a lower maximum user threshold (from a raw socket handling perspective). Such a person can be found in Willy Tarreau (the venerable author of the extremely high perfomance haproxy application).

Back to your design goal of being highly scalable to more cpus/cores, he recently decided to deviate from his proven strategy for the purpose of being able to handle SSL connections natively (to decrypt the packets so he can apply advanced routing rules instead of forwarding encrypted tcp packets without intelligent rules or relying on an external application for SSL termination). This should apply very well to the firewall packet inspection that you are looking for.

There is much discussion on his site and mailing list about these types of things, I suggest if you want the in depth details you should look around there and also at the source for haproxy. It would be a great place to pickup ideas for your project.

haproxy.1wt.eu