Hi! * On Thu, Jul 10, 2003 at 01:06 PM (+0200), José Carlos Cuéllar wrote:
One of my customers have asked me to set a system to use four 2Mb ADSL lines as a single 8 Mb one. My intention is to configure a Linux server to perform the task, but I haven't found any software yet (no linux distribution seems to support this feature by itself).
I've set up an experimental system to do the same thing (using two ADSL lines, not four - but this shouldn't matter) about one year ago. Here I'm going to resend the short "manual" which I posted in a discussion forum some months ago: 1) The Linux kernel (here: 2.4.9) must be compiled including the option "IP: equal cost multipath". 2) Two (or more) PPP connections must be established simultaneously, so "ifconfig" will give the following result, for exmample: | adsl-router:~ # ifconfig ppp0 | ppp0 Link encap:Point-to-Point Protocol | inet addr:217.80.100.254 P-t-P:212.185.251.17 Mask:255.255.255.255 | UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1492 Metric:1 | RX packets:389911 errors:0 dropped:0 overruns:0 frame:0 | TX packets:241549 errors:0 dropped:0 overruns:0 carrier:0 | collisions:0 txqueuelen:3 | RX bytes:523335248 (499.0 Mb) TX bytes:13618189 (12.9 Mb) | adsl-router:~ # ifconfig ppp1 | ppp1 Link encap:Point-to-Point Protocol | inet addr:217.80.100.226 P-t-P:212.185.251.17 Mask:255.255.255.255 | UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1492 Metric:1 | RX packets:419493 errors:0 dropped:0 overruns:0 frame:0 | TX packets:262371 errors:0 dropped:0 overruns:0 carrier:0 | collisions:0 txqueuelen:3 | RX bytes:560546553 (534.5 Mb) TX bytes:13812971 (13.1 Mb) 3) Before being able to set up the load balancing default route we've to delete any existing default route. So an "ip route show" will display: | adsl-router:~ # ip route show | 212.185.251.17 dev ppp0 proto kernel scope link src 217.80.100.254 | 212.185.251.17 dev ppp1 proto kernel scope link src 217.80.100.226 | 172.16.1.0/24 dev eth0 proto kernel scope link src 172.16.1.2 The last entry (beginning with "172.16.1.0/24") is the route which is bound to the "eth0" interface. This device is connected to the local area network (e.g. to a proxy server). 4) Then we set up the load balancing default route: ip route add default scope global \ nexthop via 212.185.251.17 dev ppp0 \ nexthop via 212.185.251.17 dev ppp1 So "ip route show" will display: | adsl-router:~ # ip route show | 212.185.251.17 dev ppp0 proto kernel scope link src 217.80.100.254 | 212.185.251.17 dev ppp1 proto kernel scope link src 217.80.100.226 | 172.16.1.0/24 dev eth0 proto kernel scope link src 172.16.1.2 | default | nexthop via 212.185.251.17 dev ppp0 weight 1 | nexthop via 212.185.251.17 dev ppp1 weight 1 So far it would be possible to use the load balancing router with LAN sourced traffic (of course, IP masquerading has to be set up also to achieve this). Locally produced traffic will result in a "destination host unreachable". 5) To allow also locally generated traffic we must set up two tables for policy routing: ip route add via 212.185.251.17 dev ppp0 src 217.80.100.254 table 100 ip route add default via 212.185.251.17 dev ppp0 table 100 ip route add via 212.185.251.17 dev ppp1 src 217.80.100.226 table 200 ip route add default via 212.185.251.17 dev ppp1 table 200 So "ip route show table 100" will produce: | adsl-router:~ # ip route show table 100 | 212.185.251.17 dev ppp0 scope link src 217.80.100.254 | default via 212.185.251.17 dev ppp0 And "ip route show table 200": | adsl-router:~ # ip route show table 200 | 212.185.251.17 dev ppp1 scope link src 217.80.100.226 | default via 212.185.251.17 dev ppp1 These two new generated routing tables will stay useless if we don't set two "ip rules". 6) Set up two "ip rules" to tell the system to use these policy routing tables: ip rule add from 217.80.100.254 table 100 ip rule add from 217.80.100.226 table 200 Now also traffic which is generated on localhost is routed to the internet service provider's PPP device. The ADSL load balancing router should now be reachable via both of its IP addresses (here: 217.80.100.254 and 217.80.100.226) from outside (which is important when setting up further firewalling rules). 7) To enable IP masquerading we've to set two firewalling rules using "iptables": iptables -t nat -A POSTROUTING -s 172.16.0.0/16 -d 0/0 -o ppp0 -j MASQUERADE iptables -t nat -A POSTROUTING -s 172.16.0.0/16 -d 0/0 -o ppp1 -j MASQUERADE 8) Now also our internal hosts should be able to use the Linux based ADSL router. The traffic should get balanced using these two PPP devices (ppp0 and ppp1). 9) The traffic balancing is done on a "per-route-basis", i.e. the ADSL router stores source address, destination address and the TOS field of an outgoing packet (in its routing cache). Any further packet that matches in those three information will get routed the same way (i.e. the same PPP device is used). At least OpenSSH and the standard FTP client seems to change the TOS field during a running TCP connection. That means that further packets may be sent out using the other device. I suppose that this leads to confusion (e.g. IP spoofing) when IP masquerading is used also. So those services will not function reliably. To resolve this problem we let our ADSL router set a default TOS field for each packet: iptables -t mangle -A PREROUTING -j TOS --set-tos 0x00 iptables -t mangle -A OUTPUT -j TOS --set-tos 0x00 Now also OpenSSH and FTP should be no problem when used from an internal host. After doing these steps you should have a load balancing and IP masquerading router. Of course, it is necessary to configure the rules in a way that they are set up everytime your PPP link comes up. Especially our internet service provider (German Telekom) terminates an unsed PPP connection every 15 minutes. Additionally after 24 hours of connection the line gets killed (then it does not matter if the connection is idle or not). This is done to change the (dynamically assigned) IP address with a certain frequency, so the ADSL lines are not used to run a web server, for example. But, of course, I have to keep this in mind when doing my configuration. So the last (outstanding) step is to set up scripts which are triggerd by "/etc/ppp/ip-up" and "/etc/ppp/ip-down". I hope that you are able to set up load balancing via two ADSL connections using this little tutorial. If you have further questions or any corrections or additions, feel free to contact me. Bye! Steffen