Before we begin, let us take a look at the picture below.
This is a very simple illustration of where in the TCP/IP stack, the two security protocols operate. This is very instructive since our discussion can be easily smoothened by understanding this key aspect.
In short, SSL requires applications to be modified as it operates above the TCP layer and this happens in user space in linux and other OSes. Whereas IPsec works seamlessly no matter what application and what protocol the application uses. ICMP traffic, UDP traffic and TCP all are protected by IPsec without the user or application developer worrying about it.
Whereas SSL involves a certain degree of user interaction. One has to verify certificates, their validity, expiry date and so on.
There is another key difference between the two security protocols.SSL protects traffic end to end whereas IPsec is usually deployed in tunnel mode in which only the edge of your network and beyond is protected by IPsec.
It is interesting to see how technical differences especially which layer a particular protocol operates influence ease of deployment.IPsec is a popular method to run corporate VPNs and gives a somewhat complete security solution.
Whereas you have to do special things to secure your e-mail traffic. For instance, you have to run POP3/IMAP as well as SMTP over SSL. And it is very difficult to secure UDP traffic. There is a new protocol called DTLS(Datagram Transport Layer Security), but it is not yet mature.
Banks and other financial institutions use SSL heavily. It puts a big load on the processing on the server side as it has to handle multiple simultaneous SSL sessions and the handshake phase can be really stressful even for modern CPUs.
Usually custom hardware is used for SSL offloading for busy servers. SSL handshaking involves public key crypto and such complex math operations cannot be parallelised or sped up easily.
And thanks to the nature of HTTP traffic, we are mostly dealing with short bursts of traffic in which TCP connections come and go. So the relatively inexpensive symmetric key operations used for SSL traffic once the handshake is over do not last long.
IPsec was originally envisaged as a security mandate for IPV6 but today it is popular as a secure VPN solution. It works very differently from SSL though the core of any security protocol involves an authentication protocol, an encryption algorithm and an integrity algorithm.
IPsec involves a key exchange phase which is fairly complex and interoperability between multiple IPsec implementations is still a hairy issue. This has led to the popularity of SSLVPNs though often the cure is worse than the disease.
stunnel is a nice little program that can be used to SSL enable your application without making a single line of code change. It can also be used to run PPP tunnels securely.
Linux 2.6.x kernel comes with full IPsec support and several other cryptographic primitives. And there are helper scripts in popular linux distros that will help you setup IPsec.
The only other popular security protocol I left out is ssh but that is used for a purpose completely different. One can tunnel arbitrary TCP traffic through ssh, but usually this is done by port forwarding.
SSHVPNs are also a good way to quickly setup a secure network. This is especially useful in WiFi deployments.
Since WiFi security protocols are not upto snuff, people usually run IPsec for security. This is more relevant since you don't want your neighbor to suck your bandwidth, do you?