suse@karsites.net wrote:
Hi Markus.
Hi Keith!
Interesting topic.
yes it is. do your remember that you lead me 10 months ago in this direction? i asked on the list for an interesting security topic for my diploma work and you told me about an iptables module to try to limit (D)DoS attacks. i invested 4 months to learn about DDoS attacks and tried to do a concept to block them. at the beginning everything looked quite simple, count the number of ip packets for each ip and so on. but after a while i had to admit that it wasn't at all as simple as i thought. if somebody is interested in why i think it is not possible with this attempt i can write more on it. ok, so i had to change my project a little bit. cause the (D)DoS topic is very interesting i decided to stay on it and do it for web servers. the main advantage over the previous work is that it is on the application layer and so i don't have to cope with spoofed packets were every packet is coming from a different ip address and there is no real chance to block such an attack.
Your idea seems very handy for doing forensic analysis, after a HTTP-DoS/DDoS attack.
I think that IPTables firewall could be used to help limit or prevent such attacks from occuring.
yes, this is already done. the system works like this. after the system is trained, it is reading continuously from the apache access logfile (or whatever web server can output combined or common format). The analysis happens with a delay of about 1 second. if a very implausibly or suspicious behavior of a client is detected, this ip address gets temporarily blacklisted. This blacklist is implemented via an apache module. if this client makes its next request it gets an HTTP error, explaining that he violated our policy and should now go to drink a coffee. when he does it (waits a little bit), he will be allowed to continue surfing on our server. if more requests knock on our door form this ip, we conclude that there's no human being behind this requests and block this ip on the firewall. i wrote a simple script which allows to do time based rules for iptables, so rules get removed automatically after some time.
There is a development library for the IPTables packet filter, that allows a user to write loadable modules for the packet filter.
I think it should be possible to write a module that will que incoming packets in userland memory. The packets can then be inspected for certain clues that would be indicative of a HTTP-DoS attack.
at the beginning of the project i wanted to write exactly such a module (cause netfilter hacking is very cool . the problem is that you are in the wrong place (my opinion) on the firewall. i found out that i would have to write a hole proxy that i can analyse the requests. its by far not enough to just look at the ip packets with the GET, POST or whatever request type in it. you have to consider fragmented ip packets and you have also to look at the response from the web server. the next problem is, that if your server supports SSL connections, your out of business (or you have to do man in the middle). at the end it was to much work to write a very fast proxy that runs on the firewall and is able to implement all HTTP standards without being an own security weakness. the next idea was to write an apache module which does the analysis. a company offered my ssh access to their web server. the problem was that the web server had to run at all times and so to restart the server all the time i compiled my apache module wasn't an option. the third idea, is now the one that gets implemented. it just works on the logfiles. this is not the fastest solution but the one with the cleanest separation. one very big advantage of this layout is, that the system can be trained with old logdata from many days in just a few minutes.
DDoS may be a bit more trickier to detect, as the source IP's will be varied, but even so, there may still be a very high number of new connection requests coming, in a very short time, from the same source IP, which would indicate a possible DoS or DDoS attack underway.
until now DDoS attacks aren't handled explicitly. the process implemented to detect attacks looks at the behavior of the request. it tries to find inhuman behavior. for example it is uncommon that a user makes every second 10 requests to always the same URL over a minute on the other side it is common that a user makes 100 requests to 100 URL s in 2 seconds (and then waits for some time) because he downloads a page with many images on it. in the training phase the program tries to find out what the common behavior for a web server is. i won't have time to implement something for DDoS attacks in this diploma work (cause it ends in a few weeks) but the link from martin mcleod brought me to an idea. the problem with DDoS attacks is that a single attack source isn't harmfully but a lot of them are. so i could try to classify the behavior of all clients. if a lot of clients have the same classification simultaneously and are using together a lot of resources, all of them get looked out....
The user written module should then be able to generate and add new rules to the IPTables firewall, to block such DoS/DDoS attacks.
After a certain amount of time, the user written module should then be able to remove those added rules from the firewall packet filter.
I suppose you would call this adaptive or intelligent firewalling, as the firewall adapts itself in response to what it sees in the INPUT chain.
well this sounds good -> adaptive or intelligent, will use this in my documentation
I need to write a white paper on this, and make it available for all to read, and hopefully someone will take up the idea and develop it into something functional!
i'm always having an open ear for your ideas!
Kind Regards - Keith Roberts
cu markus