Listmates:
On a couple of occasions today, I have had machine gun speed googlebot
crawling all over my site. tcpdump shows the culprit as:
17:21:51.123736 IP nirvana.3111skyline.com.http >
crawl-66-249-71-154.googlebot.com.46612: . ack 260 win 54
17:21:51.126689 IP nirvana.3111skyline.com.http >
crawl-66-249-71-154.googlebot.com.46612: P 1:982(981) ack 260 win 54
The IP 66.249.71.154 is google alright, but who knows if it is spoofed. I
killed it with everything I could think of:
/etc/apache2/default-server.conf
allow, deny
allow from all
deny from 66.249.71.
/srv/www/robots.txt
User-agent: *
Disallow: /
/etc/hosts.deny
ALL : 66.249.71.
Which has slowed the query/response traffic down to a trickle monitoring with
tcpdump:
[17:15 nirvana/var/log/apache2] # tcpdump -i eth0 not host alchemy and net 66.249
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
17:21:51.031714 IP crawl-66-249-71-154.googlebot.com.46612 >
nirvana.3111skyline.com.http: S 3352533987:3352533987(0) win 5840
17:21:51.031764 IP nirvana.3111skyline.com.http >
crawl-66-249-71-154.googlebot.com.46612: S 3696639852:3696639852(0) ack
3352533988 win 5792
17:21:51.123273 IP crawl-66-249-71-154.googlebot.com.46612 >
nirvana.3111skyline.com.http: . ack 1 win 92
17:21:51.123713 IP crawl-66-249-71-154.googlebot.com.46612 >
nirvana.3111skyline.com.http: P 1:260(259) ack 1 win 92
17:21:51.123736 IP nirvana.3111skyline.com.http >
crawl-66-249-71-154.googlebot.com.46612: . ack 260 win 54
I get 2-3 packets a minute here which is no big deal. Earlier, the screen
would just scroll continually and internet speed was taking a *big* hit.
So what is the consensus of the brain trust as to what I'm looking at and what
this is? Legit or something I should be concerned with? Any other defensive
measures I should take?
--
David C. Rankin, J.D.,P.E.
Rankin Law Firm, PLLC
510 Ochiltree Street
Nacogdoches, Texas 75961
Telephone: (936) 715-9333
Facsimile: (936) 715-9339
www.rankinlawfirm.com
--
To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org
For additional commands, e-mail: opensuse+help@opensuse.org
