Welcome to Linux Forums! With a comprehensive Linux Forum, information on various types of Linux software and many Linux Reviews articles, we have all the knowledge you need a click away, or accessible via our knowledgeable members.
Find the answer to your Linux question:
Site Navigation
Linux Forums
Linux Articles
Product Showcase
Linux Downloads
Linux Hosting
Free Magazines
Job Board
IRC Chat
RSS Feeds
Free Publications


This is the rest of the letter that Dr. UN*X is sending to Adrian about his question onto how can people connect to his system using the two DSL lines connected to it, read on to see the rest of the letter.

The first part of this article can be found here

Two networks = twice as much fun

Part 2. Routing two for DSL lines

A recap: I am responding this letter:
Dear Dr. UN*X,

I am currently using my Linux box as a server on the Internet, and it's all working fine. The installer program configured the network interface for me. I am currently using a single DSL line but now I have added a second line and I want remote sites to be able to connect to my system using either DSL line.

I have already installed a second network card, but the system does not see it. I checked the card maker's site and there are no Linux drivers for it!

I don't need to do anything fancy (yet) like load balancing, I just want to be able to support two outside IP addresses.

Where do I go from here?

--Adrian

Hopefully you were able to use part one of this article. If so, you now have a functioning network connection on each of your two DSL lines.

Adrian, you are setting up two lines so that your web server can be reached from two different DSL connections. You didn't say why in your email, but I am guessing you want redundancy and possibly more bandwidth to your server. Round robin DNS would be a simple approach to take once routing over both lines is working. Using multiple DSL lines is not as effective for increasing bandwidth for a single desktop system; I will talk about that later.

For the impatient

It has taken me a lot of work to get to to the point where I am comfortable working with Linux advanced routing. Let's see if I can sum things up for those of you who already understand basic routing. Then you can skip ahead to the section "Adding the second line"

With basic packet routing, there is only one routing table. The kernel choses the path for each packet by examining the destination address of that packet.

With advanced routing, you can create as many routing tables as you want using 'ip route' commands. By using 'ip rule' commands, you teach the kernel which routing table to use based on the source address of each packet. 'Advanced routing' is also called 'iproute2' or just 'iproute' in the Linux world.

That is it in a nutshell.

To get started with advanced routing you don't need to know anything about 'netfilter' (aka iptables). Netfilter can be used to build firewalls and to re-write the addresses in packets (NAT and MASQUERADE). Netfilter is a topic for another day.

Before we begin

Your system probably already has advanced routing built into the kernel; all the distributions that I use come with it. To configure it though, requires installation of a package of utilities so that you can configure the kernel.

Should you be building your own kernel, it needs to have the 'IP: Advanced router' and 'Network packet filtering' options turned on.

Whether the required utility package is installed or not depends on your distribution. The package is usually called 'iproute'. If you have the 'ip' command (look for the file '/sbin/ip') you already have it. Otherwise install the iproute package using the command appropriate for your distribution. {mospagebreak title=Just what is routing?}

Just what is routing?

I like to compare network packet routing with the post office. Let's say that you and a friend play chess by sending postcards to each other. Each postcard has a source and a destination address (From: and To: addresses) and a small amount of information, the next move in your game. You send a postcard with your move on it, then you wait for a reply to come back with your friend's next move. The game progresses as a series of transmissions and replies; the whole thing routed through any number of post offices without your knowledge. Under ordinary circumstances, to tap into this vast network, all you have to need to know are the addresses and how to find your mailbox.

Each postcard is like a network packet. With the postal system, you could send a letter if your information doesn't fit on a card. With a network, you just send more packets instead because you can't (normally) change the packet size; your operating system handles the task of breaking down large messages into small packets for transmission; likewise it reassembles incoming packets for you. This is what happens when you request a web page; you enter one address (the URL) and the network coordinates the exchange of hundreds of packets unbenownst to you. To learn about routing though, you can ignore this fact and concentrate only on the routing of a single packet.

After you drop your postcard into a mailbox, a system for routing it to its destination takes over; the reply comes back the same way. You only care about this routing mechanism when something changes, for example, if you move to a new address and want things forwarded. Otherwise it all remains invisible to you.

With the Internet, you normally only deal with one connection to this invisible network; the router provided to you by your Internet Service Provider (ISP) is your connection.

If you have one computer and one network connection, this is just about all you need to know about routing. Any packet leaving your computer heads out to the Internet, and any reply coming back from the Internet is destined for your one computer.

More complexity

Continuing with the postal analogy, if you work in the mailroom for a large company, you still generally only have one connection to the outside world, but there is an internal mailbox for each employee. As the clerk in the mailroom, you decide based on the To: address to either route a letter internally (delivering it to its local mailbox) or as before, you throw it into an outgoing pile for your connection to the world.

When you add a second computer in your office, each of your computers will have addresses indicating they are connected to the same LAN (local area network). The routing table in each of your computers has to know the correct LAN address range so that they can talk to each other.

With additional equipment, network complexity grows and things become progressively more complicated and much more interesting (to me anyway); there can be many routes in your computer's routing table. We are not getting into that here.

In the simple set up we are doing here, you only have to recognize two types of address, internal (LAN) or external (Internet). {mospagebreak title=Where do the IP addresses come from?}

Where do the IP addresses come from?

You used to have to set up ip addresses, netmasks, gateways, and name resolvers all manually. Refer to any basic networking text to learn more about this aspect.

These days DHCP (Dynamic Host Control Protocol) is very popular. Let's assume that a DHCP server (typically inside your Cable/DSL modem) and issues all the network information required. The network engineer at your upstream provider has configured the modem for you so that it will work with their network.

With a typical home setup, you are assigned just one IP address and if you have more than one computer at home, you have to play games (NAT or MASQUERADE) to share the one IP address. A typical small business connection might have 5 IP addresses assigned statically so that you can publish them; a sixth address in the range lets you communicate with the router in your DSL modem.

For a 6 address block, you'd be assigned a netmask with 29 bits (255.255.255.248 in "dotted decimal" format). You don't need to know what a netmask is right now, but it divides the address into two parts, network and local. In the postcard analogy, a p.o. box number would be the local part of your address.

Let's say your first DSL provider gives you the address range from 63.63.63.1 to 63.63.63.6; your computer is at 63.63.63.1; your DSL router is at 63.63.63.6; and the remaining 4 addresses are currently unused. {mospagebreak title=Revealing your routing tables}

Revealing your routing tables

From a command line on any Linux system, you can see the existing routing table by simply typing 'route' at the prompt. (Type '/sbin/route' if /sbin is not in your path.) Your routing table will be similar to this:

# route
destination     gateway		interface
63.63.63.0     	*		eth0
default		63.63.63.6	eth

Advanced routing commands are issued as arguments to the 'ip' command. To see the routing table with iproute2, you can use the long version "ip route show table main" or the shortcut version "ip route" like this:

# ip route
63.63.63.0/29 dev eth0 proto kernel scope link src 63.63.63.1
default via 63.63.63.6 dev eth0

Recall that in our simple example, all ip addresses are considered to be either 'local' or 'not local'. For this network, a local address will be in the range from 63.63.63.1 to 63.63.63.6. A non-local address is just anything else.* It's traditional to assign the router the highest address in the range (ending in 6 for this network).

The first line in the routing table says that if the address is local, your computer puts the packet right out onto the ethernet wire using interface /dev/eth0. The only other device on your local network right now is the DSL router at 63.63.63.6. You can probably use a browser to bring up the router's built-in web-based configuration interface by connecting to http://63.63.63.6/

When you want to surf over to linuxforums.org, the packets need to go to the Internet. You computer will use DNS to translate the name linuxforums.org into its IP address (67.15.52.42). Your computer sees that address is not local, so it uses teh 'default' entry in the routing table and sends the packet to your DSL router.

In turn, the DSL router looks at the destination address, and sees that the data is destined for somewhere on the Internet. The DSL router then pushes the packet out its DSL connection to the next router upstream which is at your ISP. Just as with the postcard analogy, it's now out of your hands; more upstream routers send it on its way to the linuxforums.org server.

When the response comes back from linuxforums.org, there is a destination address of 63.63.63.1 in the packet, so when it hits your DSL router it will pass the packet onto your LAN. Your desktop computer sees the packet on the LAN and picks it up. The round trip connection is now completed. It takes many such exchanges of packets to pass all the data required for just a single page.

If you have a web server running on your local computer, everything works pretty much the same way, but the traffic flows the other direction. Requests for pages come in from the Internet and your web server responds by sending pages back out via the DSL router. Usually you arrange to get your ip address into a DNS server somewhere so that you can publish a URL for your server with a friendly name like "http://myveryownserver.org" but you could just as easily not bother with that and just tell all your friends that your URL is http://63.63.63.1/.

When a friend surfs to http://63.63.63.1/, your web server will receive the request with a source address pointing back to your friend's computer. Thus the reply will be routed back out via your one and only default router.

* Yes, I know there is another internal network on the address 127.0.0.1. It's called the loopback interface /dev/lo. But let's not worry about it here, okay? {mospagebreak title=Adding the second line}

Adding the second line

Still with me? It should have been straightforward so far; adding the second line is when things get difficult.

After installing the new network card and getting it connected to DSL #2, you find that it's been assigned the address of 30.31.32.1 and the new DSL router is at 30.31.32.6.

Unless you made other changes, your system will probably continue to use DSL #1 as the default route and your routing table will now look like this. (I have also seen systems use the second line as the default router, I think it might be dependent on the distro.)

# ip route
63.63.63.0/29 dev eth0 proto kernel scope link src 63.63.63.1
30.31.32.0/29 dev eth1 proto kernel scope link src 30.31.32.1
default via 63.63.63.6 dev eth0

If you are sitting at the console of your machine and your personal web server is configured to listen to all available interfaces, you can bring up a web browser on the console and surf to either http://63.63.63.1/ or http://30.31.32.1/; you will see your home page both places.

But what happens when your friend on the Internet tries to hit your page at its new address http://30.31.32.1/ ? The request comes in via DSL router #2, which passes it onto your new ethernet line (/dev/eth1).

The problem is that the system does not know how to route packets back to your friend's computer. Because no connection can be established, no page gets sent back.

On your computer, you can use the ping command to test connections.

ping 63.63.63.1  test your own network interface on eth0
ping 30.31.32.1  test the interface on eth1
ping 63.63.63.6  test the router on dsl 1
ping 30.31.32.6  test the router on dsl 2
ping yahoo.com   see if you can reach the Internet

In each case you will see something similar to this:

# ping 63.63.63.6
PING 63.63.63.254 (63.63.63.254) 56(84) bytes of data.
64 bytes from 63.63.63.254: icmp_seq=1 ttl=150 time=0.480 ms
64 bytes from 63.63.63.254: icmp_seq=2 ttl=150 time=0.481 ms
64 bytes from 63.63.63.254: icmp_seq=3 ttl=150 time=0.471 ms
.
.
.
(Hit control-c to end the test)

--- 63.63.63.254 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.471/0.477/0.481/0.018 ms

All these tests should still work without changing routing, including the one that hits yahoo.com out on the Internet. Traffic bound for the Internet continues to use the default route on DSL #1. {mospagebreak title=How can I have two default routes?}

How can I have two default routes?

This is where advanced routing comes into play (finally!). With advanced routing, you can have as many routing tables as you need; in this case you need to add just one for the new DSL line.

First, add a name for the new routing table to the file /etc/iproute2/rt_tables. You can append it to the file with command "echo 2 dsl2 >> /etc/iproute2/rt_tables".

# echo 2 dsl2 >> /etc/iproute2/rt_tables
# cat /etc/iproute2/rt_tables list the file contents
#
# reserved values
#
255     local
254     main
253     default
0       unspec
#
# local
#
#1      inr.ruhep
2 dsl2                          the line you just added

Earlier I mentioned that typing "ip route" is a shortcut for the longer command "ip route show table main". Well, to list the new routing table you have to use the long form: "ip route show table dsl2" If you enter this command right now, you will see the new table is empty.

You really only need to add the new default route to the new table; the old "main" table will continue to handle everything else. You will see why in a minute. Once again, here is the existing "main" table.

# ip route show table main
63.63.63.0/29 dev eth0  proto kernel  scope link  src 63.63.63.1
30.31.32.0/29 dev eth1  proto kernel  scope link  src 30.31.32.1
default via 63.63.63.6 dev eth0

Add the new default route to table dsl2 and then look at the (short) table.


# ip route add default via 30.31.32.6 dev eth1 table dsl2
# ip route show table dsl2

default via 30.31.32.6 dev eth1    the whole table is just one line

But the new table is not used yet!

You need to learn one more command, "ip rule"

A routing table tells where packets should go (its destination). You need to be able to tell the kernel to use a different table, based on where a packet is from (its source address).

The existing ip ruleset is very simple, look at it now.

# ip rule
0:      from all lookup local
32766:  from all lookup main
32767:  from all lookup default

You need to add just one new rule:

# ip rule add from 30.31.32.1 lookup dsl2 prio 1000

This command says "add a rule" to handle the case when a packet has a "from" pattern of "30.31.32.1"; use the routing table called "dsl2", and assign the rule a priority level of "1000". Now relist the rules. In this example, the "pattern" only needs to match one address but if you build a Linux router, you could set patterns that would match different sets of addresses.

# ip rule
0:      from all lookup local
1000:   from 30.31.32.1 lookup dsl2
32766:  from all lookup main
32767:  from all lookup default

The kernel searches the ip rules in order, starting with the lowest priority and continuing through each rule and routing table until the packet has been routed successfully.

Your default ruleset will always have a 'local' table with 'all' as the match pattern. The local table handles traffic that is supposed to stay on the local machine, and broadcast traffic.

Our new rule comes next, with a priority of 1000. I picked this number to make adding other rules before and after ours easy later on.

After our rule comes the 'main' table, which is the one that is manipulated by the old 'route' command. Finally comes the 'default' table. I don't know the official purpose of default, it's empty on all the systems I have set up. There is a 'default' route in the table 'main', so no traffic ever gets to the table 'detault'.

Caveats

When you are playing with multiple routing tables, you have to remember to add the 'table' portion to the command. I have only forgotten about 1000 times now. It can be mystifying when rules change in the wrong table (main). And of course, you are sure to confuse things when learning and lock yourself out if you are remotely logged in. The changes happen FAST. Use a console connection.

Another tip. Routes are cached. This means that if you update a routing table and nothing seems to happen, don't get frustrated, just flush the cache. You can make several changes at once and then flush the cache at the end so that the changes effectively all happen simultaneously. This is handy when working on a running router.

The flush command I use is "ip route flush table cache". Be very careful with the flush command!! Enter it wrong and you will remove all routing rules, instantly cutting off your networks. {mospagebreak title=Making the new commands sticky}

Making the new commands sticky

Once the new routing table and the new rule are in place, packets coming in on the new DSL line will be routed back out the same line and all is well. But a reboot now will reset things to the previous state. On Trustix, I put the commands in a script in /etc/sysconfig/iptables-ipv4.d/ called "routing.sh" and make it executable; any executable script in this directory is run at boot time. I think this will work on other relatives of Redhat including Fedora and SUSE. On Ubuntu and its Debian-based cousins, you can put networking scripts in the /etc/network/ifup.d directory, but you create a separate script in /etc/init.d that can set up routing and firewalling and set it to run after /etc/init.d/networking. I haven't done much work with complex routing setups on Debian, as I prefer to use Trustix on servers. Refer to the update-rc.d man page on Debian systems.

Load balancing

When I originally embarked on learning advanced routing, it was to balance traffic from a LAN with around 300 nodes over several DSL lines. I am still working on it. If you want more information on load balancing see the iproute manual, especially section 4.2. (See "resources" section below.)

If you are thinking you will get twice the bandwidth on your desktop system if you add a second line, think again. Load balancing won't work very well. You can do a form of round-robin load balancing; each new outbound connection is routed to the next available line. The problem with this approach is that the route for each connection becomes associated with that one line (because the routes are cached). If you start a download from a site, all traffic will come over that one line; your download speed will be exactly the same whether you have one connection or 50. If you start a second simultaneous download from the same site, it too will use the same DSL line.

The only way this form of balancing can help is if you have many users all accessing many different sites through a central router. This is the approach that I am working on now.

I hope this article helps you to better understand the relationship between the ip rules and the ip routing tables. It seems simple to me now but it took Dr. Un*x many hours to figure it out. {mospagebreak title=Further resources}

Further resources

In learning about routing for Linux I found this book to be very useful:

Linux Routers, A Primer for Network Administrators by Tony Mancill Publisher: Prentice Hall ISBN 0-13-009026-3

Linux Advanced Routing and Traffic Control (iproute) manual: http://lartc.org/

Linux documentation project networking howto: http://www.tldp.org/HOWTO/NET3-4-HOWTO.html

The Debian reference manual has a chapter on networking: http://www.debian.org/doc/manuals/reference/ch-gateway.en.html

Commands

Tcpdump

The 'tcpdump' command is your best friend when it comes to setting up and debugging network problems.

ping

Ping is always there for you, on any Linux or Windows system. It won't always work, sometimes firewall rules block it. But using it with tcpdump is invaluable.

Suggested use: Set up one system to ping the destination, and run 'tcpdump -n -i interface icmp' on the destination system and/or a router system in between. The interface should be something like eth0 or eth1, the '-n' turns off DNS name lookups, and 'icmp' limits the traffic dumped to icmp protocol, which is what ping uses.

It's very satisfying to see ping responses start coming back when you finally get the routing set correctly.

Traceroute

The 'traceroute' command will give you a blow by blow account of the path between your machine and a remote machine. Most Linux distros don't install it by default; install the package named 'traceroute'. Just type 'traceroute' followed by a remote hostname. You will see a listing of the route packets have to take to get to the remote location. (If you only see lines with * * * there is probably a firewall in the way.)

MS Windows has the same command renamed 'tracert'.

 
Rate This Article: poor excellent
 
Comments about this article
Great intro - but I've having an annoyi
writen by: brighton36 on 2006-03-18 00:52:00
I've been using a multipath routing setup for my office for a while now. SO far, its been largely ok. The problem is that with some types of traffic my connections that should be established, oddly cease to be . SSH and IM are the two biggies. Its very frustrating. My guess is that the route tables are cleaned up, and the existing , established connection is attempted to connect out the alternate line that it was previously set at. Does anyone have this problem? Any ideas as to how to fix this? I've seen a number of other people ask in different forums, but no great solutions have been forthcoming.
RE: Great intro - but I've having an annoyi written by brighton36:
good, simple info
writen by: richard on 2006-03-27 20:36:27
Good to see a simple exlanation of what's going on. I'm setting up two ADSL connections to service my network through a router running FC4. I've got the routes part working (i hope) but I'm wondering how I can firewall both connections. I get the second ADSL modem today so I can see if the routing is working but I would appreciate some help on what I need to do re the firewall. I'm currently using firewall=iptables which is working fine but only has settings for one external and one internal interface. Can I set up a second external interface in the one script and duplicate the rules as required or should I set up a second firewall script? Any tips welcome
RE: good, simple info written by richard:
thanks!
writen by: kris on 2006-04-07 18:16:47
I just want to thank you people for writing so comprehensive and knowledgeable. All the other guides have been great; will read this one in a bit. thanks for the effort! :)
RE: thanks! written by kris:
linux driver
writen by: harmesh on 2009-08-27 23:04:43
sir,
where can i find the source code of linux driver for cctv camera
RE: linux driver written by harmesh:
Video Converter
writen by: xubeibei on 2010-07-16 02:13:36
I am really thanks for your so long and reasonable articles for us. And thanks for your advise about the book. I think the book must be useful to us and if I have time I will go to read it.
http://www.convertvideo.biz/
RE: Video Converter written by xubeibei:
impressive article
writen by: nathanroben on 2010-10-22 18:57:32
impressive article packed with lots of info, thanks a lot
RE: impressive article written by nathanroben:
sfgerg
writen by: sdfweger on 2011-03-19 06:46:38
Click on our website:

------ http://www.fullmalls.com ------

Website wholesale various fashion shoes, such as Nike, Jordan, prada, also includes the jeans, shirt, bags, hats and decoration.
Personality manufacturing execution systems (Mes) clothing,
Grab an eye bag coat tide bag

Free traffic

accept the paypal

SOCCER JERSEY 16US

jordan shoes $32

Christan Audigier bikini $23

Ed Hardy Bikini $23

Sunglass $15

COACH_Necklace $27

handbag $33

AF tank woman $17

------ http://www.fullmalls.com ------
RE: sfgerg written by sdfweger:
project
writen by: ankit288 on 2013-08-09 11:21:31
Sir,
Currently i am in 4th year ,doing bachelors in technology (B.Tech) in computer science. Please suggest me some major project in linux . I have done RHCE , RHS333 redhat certifications..
Please reply aas soon as possible.
Thank You
RE: project written by ankit288:

Comment title: * please do not put your response text here