[heroes] Re: forums migration update [2]
Per Jessen wrote:
http://forum.infra.opensuse.org/ Unfortunately you will only have access if you are on our vpn, I apologize.
I have just this morning switched over to the publicly accessible address: https://forums-nbg.opensuse.org/ I have enabled a setting to support multiple domains, but I am not sure how well vBulletin really works with that, it seems a bit mixed up. -- Per Jessen, Zürich (6.7°C) Member, openSUSE Heroes -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Op zaterdag 7 maart 2020 11:10:05 CET schreef Per Jessen:
Per Jessen wrote:
http://forum.infra.opensuse.org/ Unfortunately you will only have access if you are on our vpn, I apologize.
I have just this morning switched over to the publicly accessible address:
https://forums-nbg.opensuse.org/
I have enabled a setting to support multiple domains, but I am not sure how well vBulletin really works with that, it seems a bit mixed up. That looks a lot like our forums :-P. Awesome.
Did you see the encoding is not right ( look at Russian and Spanish )? Thanks !!! -- Gertjan Lettink a.k.a. Knurpht openSUSE Forums Team -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Knurpht-openSUSE wrote:
Op zaterdag 7 maart 2020 11:10:05 CET schreef Per Jessen:
Per Jessen wrote:
http://forum.infra.opensuse.org/ Unfortunately you will only have access if you are on our vpn, I apologize.
I have just this morning switched over to the publicly accessible address:
https://forums-nbg.opensuse.org/
I have enabled a setting to support multiple domains, but I am not sure how well vBulletin really works with that, it seems a bit mixed up.
That looks a lot like our forums :-P. Awesome.
Did you see the encoding is not right ( look at Russian and Spanish )?
Yep, lcp also pointed that out yesterday. I also see it in German. To explain - vB 5.6 recommends using utf8mb4 so I set that as the default for the database. However, the old tables are Latin1 which I think is what causes the problem. I would prefer not to do a mass conversion, so I have been trying to spot which table I could start with, to see if it works. Thanks for the feedback! -- Per Jessen, Zürich (8.2°C) Member, openSUSE Heroes -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Hello, just a quick update from last night - Per, Malcolm, Stasiek and I did some debugging. Am Samstag, 7. März 2020, 12:51:21 CET schrieb Knurpht-openSUSE:
Did you see the encoding is not right ( look at Russian and Spanish )?
Yes, and looking it the sqldump showed an interesting[tm] problem - the sqldump says DEFAULT CHARSET=latin1 - but if you pipe some of the INSERT lines into "file", it tells you that it's utf8. My guess is that the client charset was/is set incorrectly on the forums server in Provo. The obvious workaround is to adjust the CHARSET= lines in the sqldump so that mysql knows during the import that it gets utf8, but this opened another can of worms. One of them was an interesting error message saying ERROR 1114 (HY000) at line $linenumber: The table is full while we had more enough free disk space. In the middle of the night, I noticed CREATE TABLE vb_session [...] ENGINE=MEMORY so this table doesn't care about disk space, but hit the max_heap_table_size limit. (The switch from latin1 to utf8 means that a varchar(255) now needs 3*255 bytes.) Increasing max_heap_table_size seems to help, but I ended up with a "Duplicate entry 'menü' for key 'tagtext' later in the import. Well, at least we are making some progress ;-) (Interestingly, I didn't see this error when the sqldump still had CHARSET=latin1.) Per, as much as I like automated fixes (like replacing the charset with sed), I'm afraid you might need to let vB create an empty reference database to see the expected encoding for each field, and then adjust the sqldump accordingly :-( Please also check which database engine gets used - as much as I understand using MEMORY for the session table, I'm not sure if this makes too much sense. There should be reasons different from "someone restarted mysql" to clean the session table ;-) Also have a look at the IRC scrollback - this mail is only a rough summary. Regards, Christian Boltz -- Nimm bitte ein BSD-iges Mac OS. Denn damit braucht man kein Viagra mehr, die Heizung kommt ohne Öl oder Gas aus, nie wieder regnet es!1!! und überhaupt löst sich so vieles, ach wenn nicht alles von selbst. [Lars Müller in opensuse-de] -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Christian Boltz wrote:
Yes, and looking it the sqldump showed an interesting[tm] problem - the sqldump says DEFAULT CHARSET=latin1 - but if you pipe some of the INSERT lines into "file", it tells you that it's utf8. My guess is that the client charset was/is set incorrectly on the forums server in Provo.
Yes, I'm beginning to wonder about that too.
The obvious workaround is to adjust the CHARSET= lines in the sqldump so that mysql knows during the import that it gets utf8, but this opened another can of worms.
For now, I have just completed an import : 1) excluding vb_session 2) setting default charset and collate on the database (utb8mb4,utf8mb4?bin) 3) omitting the default charset=latin1. It still does not work 100% though.
I noticed CREATE TABLE vb_session [...] ENGINE=MEMORY so this table doesn't care about disk space, but hit the max_heap_table_size limit.
Yeah, that was a nice find.
Increasing max_heap_table_size seems to help, but I ended up with a "Duplicate entry 'menü' for key 'tagtext' later in the import. Well, at least we are making some progress ;-) (Interestingly, I didn't see this error when the sqldump still had CHARSET=latin1.)
The dup key is basically the collation thinking 'u' and 'ü' are the same. I managed to avoid that by using utf8mb4_bin.
Per, as much as I like automated fixes (like replacing the charset with sed), I'm afraid you might need to let vB create an empty reference database to see the expected encoding for each field, and then adjust the sqldump accordingly :-(
The database schema don't match ....
Please also check which database engine gets used
Previously it was myisam, but I've switched all of those to innodb.
- as much as I understand using MEMORY for the session table, I'm not sure if this makes too much sense. There should be reasons different from "someone restarted mysql" to clean the session table ;-)
scope creep :-) Thanks for the summary, Christian. -- Per Jessen, Zürich (8.1°C) Member, openSUSE Heroes -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
participants (3)
-
Christian Boltz -
Knurpht-openSUSE -
Per Jessen