Hello,
good news: after a month of hunting a) spammers and b) server admins, the needed wiki extensions to prevent and delete wiki spam finally got installed and the database updated so that those extensions can work. (Done in the english wiki, the others will follow.)
This means the wiki is editable again :-)
It also means we now have an easy way to block spam, and can mass-delete existing spam pages.
As every good news, there's also a bit of bad news - the database update cleared all cache, which means the wiki is terribly slow at the moment.
BTW: If I accidently delete something that was not spam while doing the mass deletion, please tell me ;-)
For those who are interested in details: - the Nuke and AbuseFilter extensions were installed - I just added the first spam filter - let's see how it works out (and tell me if it accidently produces false positives [1]) - I just started to delete all the spam. If someone has admin permissions on the english wiki and wants to help, head over to https://en.opensuse.org/Special:Nuke and enter something like %1_8% as page title search term. It uses MySQL LIKE syntax, as a regex the example means .*1.8.* (the "all pages" list will give you more spam page titles if you sort by creation date) - If you are interested in helping with AbuseFilter rules, see https://www.mediawiki.org/wiki/Manual:Combating_spam/AbuseFilter_examples and https://www.mediawiki.org/wiki/Extension:AbuseFilter#Documentation_and_manag... for documentation.
If you reply to this mail, please answer only to one list. Cross-posting the announcement is enough ;-)
Regards,
Christian Boltz
[1] Funnily I was the first victim of the spam filter ;-)
I didn't exclude the 'delete' action on the first attemp. When I tried to delete pages with spammy title, this was denied because the page title matched the spam filter ;-)
Lesson learned: I need to exclude the 'delete' action from the filter ;-)
Hi Christian,
Thanks to you and all involved in getting things not only fixed and but improved so we hopefully won't meet such problems again
I know it hasn't been easy, but from the sound of things we've come out on the other side of this much better equipped than we went into it ;)
Regards,
Richard openSUSE Board Chairman
On 29 March 2016 at 22:24, Christian Boltz opensuse@cboltz.de wrote:
Hello,
good news: after a month of hunting a) spammers and b) server admins, the needed wiki extensions to prevent and delete wiki spam finally got installed and the database updated so that those extensions can work. (Done in the english wiki, the others will follow.)
This means the wiki is editable again :-)
It also means we now have an easy way to block spam, and can mass-delete existing spam pages.
As every good news, there's also a bit of bad news - the database update cleared all cache, which means the wiki is terribly slow at the moment.
BTW: If I accidently delete something that was not spam while doing the mass deletion, please tell me ;-)
For those who are interested in details:
- the Nuke and AbuseFilter extensions were installed
- I just added the first spam filter - let's see how it works out (and tell me if it accidently produces false positives [1])
- I just started to delete all the spam. If someone has admin permissions on the english wiki and wants to help, head over to https://en.opensuse.org/Special:Nuke and enter something like %1_8% as page title search term. It uses MySQL LIKE syntax, as a regex the example means .*1.8.* (the "all pages" list will give you more spam page titles if you sort by creation date)
- If you are interested in helping with AbuseFilter rules, see https://www.mediawiki.org/wiki/Manual:Combating_spam/AbuseFilter_examples and https://www.mediawiki.org/wiki/Extension:AbuseFilter#Documentation_and_manag... for documentation.
If you reply to this mail, please answer only to one list. Cross-posting the announcement is enough ;-)
Regards,
Christian Boltz
[1] Funnily I was the first victim of the spam filter ;-)
I didn't exclude the 'delete' action on the first attemp. When I tried to delete pages with spammy title, this was denied because the page title matched the spam filter ;-) Lesson learned: I need to exclude the 'delete' action from the filter ;-)-- For Geralds problem "rpm -e digikam-doc" would be the solution: nobody is going to read 19.3MB of documentation anyway, given that people do not even read short README's :-P [Stefan Seyfried in opensuse-factory]
-- To unsubscribe, e-mail: opensuse-project+unsubscribe@opensuse.org To contact the owner, email: opensuse-project+owner@opensuse.org
Hello,
Am Dienstag, 29. März 2016, 22:24:16 CEST schrieb Christian Boltz:
good news: after a month of hunting a) spammers and b) server admins, the needed wiki extensions to prevent and delete wiki spam finally got installed and the database updated so that those extensions can work.
Two weeks later, I have another update - hopefully the final one on this topic.
We deleted all spam from the english wiki, and have set up filter rules (using the AbuseFilter extension) to prevent new spam.
"All spam" means: we had to delete 6612 spam files, 513 pages in the main namespace and and about 50 pages spread over the other namespaces.
I had the most fun with deleting about 5000 pages and files, and Micah (one of the Admins in Provo who is also a good Mail/IRC to SSH relay ;-) deleted the remaining 2200 spam files after cleaning up the recentchanges table (see the "Nuke" section below).
While working on this, some funny things happened - as usual when I use or touch some code the first time, it breaks into parts ;-) and someone needs to fix it.
AbuseFilter was my first victim - it worked fine for blocking spam pages, but it didn't block uploading of spam files with similar titles. It turned out that AbuseFilter used a wrong variable name for file uploads, making it impossible to filter for upload filenames. This was fixed (after I poked some people on #mediawiki) in https://gerrit.wikimedia.org/r/#/c/281234/2 I applied this fix to the openSUSE wiki.
This bug might also explain why the spammers prefer file uploads - lots of wikis around the world don't have this fix yet and therefore make it easy to upload spam files. (Dear spammers, sorry for finding this bug and making your life harder *g,d&r*)
My second victim was the Nuke extension - after deleting some of the spam, I noticed that it offered the same files for deletion over and over again, even if they were already deleted. (This happened only for files, not for "normal" pages.)
It turned out that Nuke works based on the "recentchanges" table. This works fine for normal pages because Mediawiki deletes their "create" event from recentchanges when you delete them. However, it doesn't do this with "upload" events when deleting a file. (I'll re-check this behaviour with a newer version of MediaWiki, and open a bugreport if it's still the same.)
As a workaround, we used a manual delete query on the recentchanges table. Not too nice, but it worked.
(Done in the english wiki, the others will follow.)
Micah deployed the Nuke and AbuseFilter extensions on the non-english staging wikis (for example destage.o.o), and will push them to all production wikis next week.
This means the admins of the non-english wikis will be able to use Special:Nuke and Special:Abusefilter soon. (See my previous mail on this topic for some links and instructions.)
My request from the lat mail still stands:
If you reply to this mail, please answer only to one list. Cross-posting the announcement is enough ;-)
Regards,
Christian Boltz
On Sun, Apr 17, 2016 at 3:28 AM, Christian Boltz opensuse@cboltz.de wrote:
Hello,
Micah deployed the Nuke and AbuseFilter extensions on the non-english staging wikis (for example destage.o.o), and will push them to all production wikis next week.
Nuke returns "Permission error" because "The action you have requested is limited to users in the group: Administrators." but AbuseFilter returns "Welcome to the Abuse Filter management interface."
That seems a bit inconsistent.
-
Christian Boltz -
PatrickD Garvey -
Richard Brown