On 29.07.2010 11:45, Christian Boltz wrote:
Hello,
on Mittwoch, 28. Juli 2010, Matthew Ehle wrote:
It appears that when you get towards the end of the useful search results, the search results start getting into the history and diff pages. I don't know if that's really something we want, but it should be easy to change the CSE settings to fix that.
Don't fix the search results, fix the indexing ;-)
I'd propose to create a robots.txt with Disallow: /index.php
This should keep out the page history, view source etc. out of search engines. Articles will still be listed because they don't have index.php in their URL.
You may also want to add things like Disallow: /Special:Search Disallow: /Special:Random and all its translated counterparts - http://de.wikipedia.org/robots.txt has a nice list ;-)
Hi, I think excluding /index.php is a good way to go. I added a robots.txt to the wiki sources. Matthew: To make this file available we need to change the apache rewrite conditions, I already changed that on staging. Greetings -- Thomas Schmidt (tschmidt [at] suse.de) SUSE Linux Products GmbH :: Research & Development :: Tools "Don't Panic", Douglas Adams (1952 - 11.05.2001) -- To unsubscribe, e-mail: opensuse-wiki+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-wiki+help@opensuse.org