Hello,
good news: after a month of hunting a) spammers and b) server admins, the needed wiki extensions to prevent and delete wiki spam finally got installed and the database updated so that those extensions can work. (Done in the english wiki, the others will follow.)
This means the wiki is editable again :-)
It also means we now have an easy way to block spam, and can mass-delete existing spam pages.
As every good news, there's also a bit of bad news - the database update cleared all cache, which means the wiki is terribly slow at the moment.
BTW: If I accidently delete something that was not spam while doing the mass deletion, please tell me ;-)
For those who are interested in details: - the Nuke and AbuseFilter extensions were installed - I just added the first spam filter - let's see how it works out (and tell me if it accidently produces false positives [1]) - I just started to delete all the spam. If someone has admin permissions on the english wiki and wants to help, head over to https://en.opensuse.org/Special:Nuke and enter something like %1_8% as page title search term. It uses MySQL LIKE syntax, as a regex the example means .*1.8.* (the "all pages" list will give you more spam page titles if you sort by creation date) - If you are interested in helping with AbuseFilter rules, see https://www.mediawiki.org/wiki/Manual:Combating_spam/AbuseFilter_examples and https://www.mediawiki.org/wiki/Extension:AbuseFilter#Documentation_and_manag... for documentation.
If you reply to this mail, please answer only to one list. Cross-posting the announcement is enough ;-)
Regards,
Christian Boltz
[1] Funnily I was the first victim of the spam filter ;-)
I didn't exclude the 'delete' action on the first attemp. When I tried to delete pages with spammy title, this was denied because the page title matched the spam filter ;-)
Lesson learned: I need to exclude the 'delete' action from the filter ;-)
Hello,
Am Dienstag, 29. März 2016, 22:24:16 CEST schrieb Christian Boltz:
good news: after a month of hunting a) spammers and b) server admins, the needed wiki extensions to prevent and delete wiki spam finally got installed and the database updated so that those extensions can work.
Two weeks later, I have another update - hopefully the final one on this topic.
We deleted all spam from the english wiki, and have set up filter rules (using the AbuseFilter extension) to prevent new spam.
"All spam" means: we had to delete 6612 spam files, 513 pages in the main namespace and and about 50 pages spread over the other namespaces.
I had the most fun with deleting about 5000 pages and files, and Micah (one of the Admins in Provo who is also a good Mail/IRC to SSH relay ;-) deleted the remaining 2200 spam files after cleaning up the recentchanges table (see the "Nuke" section below).
While working on this, some funny things happened - as usual when I use or touch some code the first time, it breaks into parts ;-) and someone needs to fix it.
AbuseFilter was my first victim - it worked fine for blocking spam pages, but it didn't block uploading of spam files with similar titles. It turned out that AbuseFilter used a wrong variable name for file uploads, making it impossible to filter for upload filenames. This was fixed (after I poked some people on #mediawiki) in https://gerrit.wikimedia.org/r/#/c/281234/2 I applied this fix to the openSUSE wiki.
This bug might also explain why the spammers prefer file uploads - lots of wikis around the world don't have this fix yet and therefore make it easy to upload spam files. (Dear spammers, sorry for finding this bug and making your life harder *g,d&r*)
My second victim was the Nuke extension - after deleting some of the spam, I noticed that it offered the same files for deletion over and over again, even if they were already deleted. (This happened only for files, not for "normal" pages.)
It turned out that Nuke works based on the "recentchanges" table. This works fine for normal pages because Mediawiki deletes their "create" event from recentchanges when you delete them. However, it doesn't do this with "upload" events when deleting a file. (I'll re-check this behaviour with a newer version of MediaWiki, and open a bugreport if it's still the same.)
As a workaround, we used a manual delete query on the recentchanges table. Not too nice, but it worked.
(Done in the english wiki, the others will follow.)
Micah deployed the Nuke and AbuseFilter extensions on the non-english staging wikis (for example destage.o.o), and will push them to all production wikis next week.
This means the admins of the non-english wikis will be able to use Special:Nuke and Special:Abusefilter soon. (See my previous mail on this topic for some links and instructions.)
My request from the lat mail still stands:
If you reply to this mail, please answer only to one list. Cross-posting the announcement is enough ;-)
Regards,
Christian Boltz
On Sun, Apr 17, 2016 at 3:28 AM, Christian Boltz opensuse@cboltz.de wrote:
Hello,
Micah deployed the Nuke and AbuseFilter extensions on the non-english staging wikis (for example destage.o.o), and will push them to all production wikis next week.
Nuke returns "Permission error" because "The action you have requested is limited to users in the group: Administrators." but AbuseFilter returns "Welcome to the Abuse Filter management interface."
That seems a bit inconsistent.
Hello,
[only replying to -wiki because this is a completely "technical" mail]
Am Dienstag, 19. April 2016, 16:08:13 CEST schrieb PatrickD Garvey:
On Sun, Apr 17, 2016 at 3:28 AM, Christian Boltz wrote:
Micah deployed the Nuke and AbuseFilter extensions on the non-english staging wikis (for example destage.o.o), and will push them to all production wikis next week.
Nuke returns "Permission error" because "The action you have requested is limited to users in the group: Administrators." but AbuseFilter returns "Welcome to the Abuse Filter management interface."
That seems a bit inconsistent.
Not really ;-)
If Nuke would be public, you could use it to search for pages matching a specific pattern, namespace and/or username. Maybe it could even display a list of matching pages to you, but a) that's not too useful because you can get similar list via special:allpages, special:contributions and special:recentchanges and b) the main action is to _delete_ those pages, which only admins can do.
AbuseFilter allows non-admins to view the filters and the filter log, which is useful for admins of other wikis who might want to "steal" a filter, for users who want to find out why one of their edits got accidently blocked - and for statistics fans because you can also see the matches each filter produced (unless a filter gets marked as non- public). So it's much more useful for non-admins - the only thing they can't do is editing the filters.
As a sidenote - the "Notes: (private)" field of each filter isn't as private as the label might indicate - it's visible without being logged in...
Regards,
Christian Boltz
On Wed, Apr 20, 2016 at 3:25 PM, Christian Boltz opensuse@cboltz.de wrote:
Hello,
[only replying to -wiki because this is a completely "technical" mail]
Am Dienstag, 19. April 2016, 16:08:13 CEST schrieb PatrickD Garvey:
On Sun, Apr 17, 2016 at 3:28 AM, Christian Boltz wrote:
Micah deployed the Nuke and AbuseFilter extensions on the non-english staging wikis (for example destage.o.o), and will push them to all production wikis next week.
Nuke returns "Permission error" because "The action you have requested is limited to users in the group: Administrators." but AbuseFilter returns "Welcome to the Abuse Filter management interface."
That seems a bit inconsistent.
Not really ;-)
If Nuke would be public, you could use it to search for pages matching a specific pattern, namespace and/or username. Maybe it could even display a list of matching pages to you, but a) that's not too useful because you can get similar list via special:allpages, special:contributions and special:recentchanges and b) the main action is to _delete_ those pages, which only admins can do.
AbuseFilter allows non-admins to view the filters and the filter log, which is useful for admins of other wikis who might want to "steal" a filter, for users who want to find out why one of their edits got accidently blocked - and for statistics fans because you can also see the matches each filter produced (unless a filter gets marked as non- public). So it's much more useful for non-admins - the only thing they can't do is editing the filters.
As a sidenote - the "Notes: (private)" field of each filter isn't as private as the label might indicate - it's visible without being logged in...
Thank you.
-
Christian Boltz -
PatrickD Garvey