[opensuse-wiki] Problems after Wiki upgrade...

older
[opensuse-wiki] Re: [opensuse-web]...

G G

1 Dec 2011 1 Dec '11

20:49

Hello Everyone, i'm hawake, the coordinator of the Italian traslation of the Wiki. I'm here to report some troubles we are having using the wiki: - [0] http://it.opensuse.org/Portal:Wiki/Novità ( a page we used like showcase ) has disappeared completely... - the <gallery> tag is now messed up (example: http://it.opensuse.org/KDE) ; - now, comparing two versions of the same page is more difficult because of non highlighted changes with different colors; - the link [edit] for each section, now on the left, messes the titles layout complicating the readability. Does someone have an idea about how to solve the <gallery> issue? And do you know why pages like [0] has disappeared completely (no history...)? Thanks for help! Bye hawake -- Linux user number 433087 Linux registered machine number 351448 http://counter.li.org/ -- To unsubscribe, e-mail: opensuse-wiki+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-wiki+owner@opensuse.org

Show replies by thread

Matthew Ehle

1 Dec 1 Dec

23:14

Matthew Ehle Web Engineer IS&T Mobile Phone: (801) 358-1655 mehle@novell.com

...

...
...
G G 12/1/2011 1:49 PM >>> Hello Everyone, i'm hawake, the coordinator of the Italian traslation of the Wiki. I'm here to report some troubles we are having using the wiki:

- [0] http://it.opensuse.org/Portal:Wiki/Novità ( a page we used like showcase ) has disappeared completely... It has not actually disappeared. The update script has messed up all the UTF8 titles, so they only appear to be gone. This was apparently a known issue, but no one has a fix for it yet. See http://www.mediawiki.org/wiki/Thread:Project:Support_desk/UTF8_Problem_with_....

...

- the <gallery> tag is now messed up (example: http://it.opensuse.org/KDE) ;

There are some theming issues that we are working on. The new resource loader feature of MW is having some trouble with our theme.

...

- now, comparing two versions of the same page is more difficult because of non highlighted changes with different colors; - the link [edit] for each section, now on the left, messes the titles layout complicating the readability. I believe they are all the same theming issues mentioned above.

...

Does someone have an idea about how to solve the <gallery> issue? And do you know why pages like [0] has disappeared completely (no history...)? I'm trying to figure out how to fix the broken names, and some of the contributors are working on a fix for the skin. I would suggest not trying to rebuild the page, as it will probably get overwritten when we try to fix it.

Thank you, Matt

G G

2 Dec 2 Dec

09:39

Hi Matt, thank you for answer! I'll wait news about those problems here. Bye hawake -- Linux user number 433087 Linux registered machine number 351448 http://counter.li.org/ -- To unsubscribe, e-mail: opensuse-wiki+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-wiki+owner@opensuse.org

C

10:12

On Fri, Dec 2, 2011 at 00:14, Matthew Ehle wrote:

...

...
...
...
G G 12/1/2011 1:49 PM >>>

...
Hello Everyone, i'm hawake, the coordinator of the Italian traslation of the Wiki. I'm here to report some troubles we are having using the wiki:

- [0] http://it.opensuse.org/Portal:Wiki/Novità ( a page we used like showcase ) has disappeared completely... It has not actually disappeared. The update script has messed up all the UTF8 titles, so they only appear to be gone. This was apparently a known issue, but no one has a fix for it yet. See http://www.mediawiki.org/wiki/Thread:Project:Support_desk/UTF8_Problem_with_....

...
- the <gallery> tag is now messed up (example: http://it.opensuse.org/KDE) ;

There are some theming issues that we are working on. The new resource loader feature of MW is having some trouble with our theme.

...
- now, comparing two versions of the same page is more difficult because of non highlighted changes with different colors; - the link [edit] for each section, now on the left, messes the titles layout complicating the readability. I believe they are all the same theming issues mentioned above.

...
Does someone have an idea about how to solve the <gallery> issue? And do you know why pages like [0] has disappeared completely (no history...)? I'm trying to figure out how to fix the broken names, and some of the contributors are working on a fix for the skin. I would suggest not trying to rebuild the page, as it will probably get overwritten when we try to fix it.

Just a thought on this... Since this recent MedaiWiki update was an upgrade from an old MediaWiki instance running on an existing/old MySQL database... is it possible that some of the tables were set to Latin1 and contained UTF-8 encoded data? I've seen this happen, and you can tie yourself up in interesting knots if you try to export a MySQL database from Latin1 to UTF-8... you end up with double encoded characters. The result is that data is stored in the database with the broken double encoded characters in the article name. When you request the article using the UTF-8 encoded characters, the MW engine thinks you are requesting a new page. If you request the page using the broken double encoded characters (assuming you can figure out what the double encoded chars are), you get the "lost" article. One way to fix this - *IF* this is what happened - is to export the tables as Latin1 (which contain UTF-8 data) instead of UTF-8 so that MySQL doesn't try to "convert" the tables, change the charset in the database dump to UTF-8 (use an editor to do a find/replace for Latin1 to UTF-8), recreate the tables in UTF-8 instead of Latin1 (you may need to tweak the create table statements in the database dump and then import your database data. See also: http://dev.mysql.com/doc/refman/5.0/en/alter-table.html in particular, this bit: ------------- Warning The CONVERT TO operation converts column values between the character sets. This is not what you want if you have a column in one character set (like latin1) but the stored values actually use some other, incompatible character set (like utf8). In this case, you have to do the following for each such column: ALTER TABLE t1 CHANGE c1 c1 BLOB; ALTER TABLE t1 CHANGE c1 c1 TEXT CHARACTER SET utf8; The reason this works is that there is no conversion when you convert to or from BLOB columns. ------------------ This also may work.. and might be a more appropriate/elegant solution if you know which columns are affected. If you've got an offline mirror that you can test the dB on, it's not a difficult test to do... if it fixes things, then you know what happened :-) and if not... well, we're back at square one again. C. -- To unsubscribe, e-mail: opensuse-wiki+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-wiki+owner@opensuse.org

Matthew Ehle

15:26

...

...
I'm trying to figure out how to fix the broken names, and some of the contributors are working on a fix for the skin. I would suggest not trying to rebuild the page, as it will probably get overwritten when we try to fix it.

Just a thought on this... Since this recent MedaiWiki update was an upgrade from an old MediaWiki instance running on an existing/old MySQL database... is it possible that some of the tables were set to Latin1 and contained UTF-8 encoded data?

Yes, you are on the right path with this.

...

I've seen this happen, and you can tie yourself up in interesting knots if you try to export a MySQL database from Latin1 to UTF-8... you end up with double encoded characters. The result is that data is stored in the database with the broken double encoded characters in the article name. When you request the article using the UTF-8 encoded characters, the MW engine thinks you are requesting a new page. If you request the page using the broken double encoded characters (assuming you can figure out what the double encoded chars are), you get the "lost" article.

One way to fix this - *IF* this is what happened - is to export the tables as Latin1 (which contain UTF-8 data) instead of UTF-8 so that MySQL doesn't try to "convert" the tables, change the charset in the database dump to UTF-8 (use an editor to do a find/replace for Latin1 to UTF-8), recreate the tables in UTF-8 instead of Latin1 (you may need to tweak the create table statements in the database dump and then import your database data.

This is exactly the path I was following last night with the stage environment. It *almost* worked. Unfortunately, we are having a few other issues that I have to look at. But yes, you are correct in that this is 95% of the problem.

...

See also: http://dev.mysql.com/doc/refman/5.0/en/alter-table.html in particular, this bit: ------------- Warning The CONVERT TO operation converts column values between the character sets. This is not what you want if you have a column in one character set (like latin1) but the stored values actually use some other, incompatible character set (like utf8). In this case, you have to do the following for each such column:

ALTER TABLE t1 CHANGE c1 c1 BLOB; ALTER TABLE t1 CHANGE c1 c1 TEXT CHARACTER SET utf8; The reason this works is that there is no conversion when you convert to or from BLOB columns. ------------------ This also may work.. and might be a more appropriate/elegant solution if you know which columns are affected. I haven't tried this yet. This actually may be a much better way to go. Thanks for the tip!

...

If you've got an offline mirror that you can test the dB on, it's not a difficult test to do... if it fixes things, then you know what happened :-) and if not... well, we're back at square one again.

I'm trying this all out on our staging wikis. I imported the production data down to rustage.opensuse.org, and that is where I am doing most of my work. I managed to get the titles to at least appear correctly, but we are having some other issues as well. I haven't given up hope though. -Matt

C

15:37

On Fri, Dec 2, 2011 at 16:26, Matthew Ehle wrote:

...

...
Just a thought on this... Since this recent MedaiWiki update was an upgrade from an old MediaWiki instance running on an existing/old MySQL database... is it possible that some of the tables were set to Latin1 and contained UTF-8 encoded data? Yes, you are on the right path with this.

Aha :-) Well, the good thing is if you know the root of the problem, it's usually fixable... maybe painful to fix, but fixable in the end.

...

environment. It *almost* worked. Unfortunately, we are having a few other issues that I have to look at. But yes, you are correct in that this is 95% of the problem.

Almost - I know that one. Up until this summer, I was the lead server admin on the OpenOffice.org wiki, and backed myself into many creative corners while doing upgrades.

...

I'm trying this all out on our staging wikis. I imported the production data down to rustage.opensuse.org, and that is where I am doing most of my work. I managed to get the titles to at least appear correctly, but we are having some other issues as well. I haven't given up hope though.

Are the other issues known? I mean.. are you still trying to figure them out, or have a fix in process? Just curious. I'm no super expert on Wikis and MySQL, but I have bumbled into some oddities in the 5 years or so I was administering the OOo Wiki. C. -- To unsubscribe, e-mail: opensuse-wiki+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-wiki+owner@opensuse.org

Matthew Ehle

15:46

...

...
...
...
C 12/2/2011 8:37 AM >>> On Fri, Dec 2, 2011 at 16:26, Matthew Ehle wrote: Just a thought on this... Since this recent MedaiWiki update was an upgrade from an old MediaWiki instance running on an existing/old MySQL database... is it possible that some of the tables were set to Latin1 and contained UTF-8 encoded data? Yes, you are on the right path with this.

Aha :-) Well, the good thing is if you know the root of the problem, it's usually fixable... maybe painful to fix, but fixable in the end.

I really hope so. I hate wasting people's hard work, even if it is just a few days.

...

...
environment. It *almost* worked. Unfortunately, we are having a few other issues that I have to look at. But yes, you are correct in that this is 95% of the problem.

Almost - I know that one. Up until this summer, I was the lead server admin on the OpenOffice.org wiki, and backed myself into many creative corners while doing upgrades.

When looking at the English wiki, I think I found the problem. Definitely an issue in the MW upgrade path, because the "new" wikis (en and de) actually do not have the problem with UTF8 page titles. Certain tables on there have a different encoding than the other wikis. Surprise, surprise :)

...

...
I'm trying this all out on our staging wikis. I imported the production data down to rustage.opensuse.org, and that is where I am doing most of my work. I managed to get the titles to at least appear correctly, but we are having some other issues as well. I haven't given up hope though.

Are the other issues known? I mean.. are you still trying to figure them out, or have a fix in process? Just curious. I'm no super expert on Wikis and MySQL, but I have bumbled into some oddities in the 5 years or so I was administering the OOo Wiki.

They are known, but there is no fix proposed. I may be the one who ends up answering those questions for the MW community. Assuming I can figure it out of course...

4542

Age (days ago)

4543

Last active (days ago)

List overview

Download

6 comments

3 participants

participants (3)

C
G G
Matthew Ehle