Hi forum admins (and fellow heroes), On behalf of the openSUSE Heroes team, I'd like to inform you that I'm working on upgrading the software stack running the openSUSE Forum server. I have set up a test server to demo the result of the upgrade: https://osforums.plexor.ch/ Please take a look at it, and give me feedback on your thoughts. As you will quickly notice, the visual appearance of the demo server is not right. That's because our custom theming on the current forum server is incompatible with newer versions of vBulletin, so it can't be reused, unfortunately. In my opinion, upgrading the production server is blocked until a new theme has been created - do you agree? (At the very least, the logo must be replaced, and more green would be nice, too.) Since I'm not very familiar with frontend technology, I'd appreciate getting some help with this. Would anyone (any community member) like to help recreate the openSUSE styling for the new forum? Also, does anyone know who created the original custom styling for the forums? Other than the loss of styling, migration of all forum content works as expected and without error, as far as I can tell. But of course, if anyone finds any migration issues, especially with forum content, please let me know. Looking forward to hear your feedback. Regards, Olav Reinert TECHNICAL DETAILS: The current software stack running the forums is: * vBulletin 4.2 * PHP 5.5 * SLES 12 SP5 The upgrade will migrate the forum server to the following stack: * vBulletin 5.6 * PHP 7.4 * openSUSE Leap 15.2 The current forum server works, of course, but is based on rather old software versions. More importantly, it isn't administered by the SaltStack configuration used to manage most other openSUSE infrastructure - and likely never will be in its current state. This makes it very cumbersome, and a bit risky, for the Heroes team to maintain, and is one of the main motivations for upgrading.
Am Di, 12. Jan, 2021 um 2:18 P. M. schrieb Olav Reinert
Hi forum admins (and fellow heroes),
On behalf of the openSUSE Heroes team, I'd like to inform you that I'm working on upgrading the software stack running the openSUSE Forum server.
I have set up a test server to demo the result of the upgrade:
Please take a look at it, and give me feedback on your thoughts.
As you will quickly notice, the visual appearance of the demo server is not right. That's because our custom theming on the current forum server is incompatible with newer versions of vBulletin, so it can't be reused, unfortunately. In my opinion, upgrading the production server is blocked until a new theme has been created - do you agree? (At the very least, the logo must be replaced, and more green would be nice, too.)
Since I'm not very familiar with frontend technology, I'd appreciate getting some help with this. Would anyone (any community member) like to help recreate the openSUSE styling for the new forum? Also, does anyone know who created the original custom styling for the forums?
Other than the loss of styling, migration of all forum content works as expected and without error, as far as I can tell. But of course, if anyone finds any migration issues, especially with forum content, please let me know.
Looking forward to hear your feedback.
Regards, Olav Reinert
TECHNICAL DETAILS:
The current software stack running the forums is:
* vBulletin 4.2 * PHP 5.5 * SLES 12 SP5
The upgrade will migrate the forum server to the following stack:
* vBulletin 5.6 * PHP 7.4 * openSUSE Leap 15.2
The current forum server works, of course, but is based on rather old software versions.
More importantly, it isn't administered by the SaltStack configuration used to manage most other openSUSE infrastructure - and likely never will be in its current state. This makes it very cumbersome, and a bit risky, for the Heroes team to maintain, and is one of the main motivations for upgrading.
I actually already had a version of our forums running using Discourse, and I think if we are gonna migrate the forums we might as well go with something open source right away ;) The snag we had, and I can't tell still how important it is, is the thing that also happens with your version of the forums: https://osforums.plexor.ch/forum/deutsch-german/hilfe-und-helfen/installatio... forums before certain date are completely broken due to changes in encoding in php decade ago. Honestly if you have the time to spend on vb5, I would rather like to see the time spent on migrating to Discourse, and I can probably help in some capacity so we get it all salted as well right away. Most important notes in that migration is that you have to use a converter script on the database because if you don't you end up with the only working posts being the ones from before 2011 or so. I used: https://gist.github.com/gschlager/bad57293004c0bfcddba and had success with everything working correctly. Additional point, since discourse maps users onto emails, we have to request SUSE IT to give us a list of usernames matching emails (because vb4 has wrong emails for usernames, I don't think it's important to explain why that is). We agreed with the Forums team on deploying at least https://github.com/sman591/discourse-nntp-bridge. Discourse will have to be set up using OIDC provided by our ipsilon instance at id.opensuse.org, and I hope we can also use the js script that embeds discourse into pages as a comments system for news.opensuse.org, so we have comments functionality there (in the future we may also want to use it for software-o-o, but that's still up in the air). LCP [Stasiek] https://lcp.world
On Tue, 12 Jan 2021 15:10:02 +0100
Stasiek Michalski
Am Di, 12. Jan, 2021 um 2:18 P. M. schrieb Olav Reinert
: Hi forum admins (and fellow heroes),
On behalf of the openSUSE Heroes team, I'd like to inform you that I'm working on upgrading the software stack running the openSUSE Forum server. <snip>
I actually already had a version of our forums running using Discourse, and I think if we are gonna migrate the forums we might as well go with something open source right away ;)
<snip> Hi All That was my understanding as well, move to new platform.... My thought is just archiving off the older posts in the 4.2 release, this becomes read only for users to access (link too etc), newer posts imported into new Forum platform (say 12-18 months)... -- Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890) Tumbleweed 20210110 | GNOME Shell 3.38.2 | 5.10.5-1-default Intel DQ77MK MB | Xeon E3-1245 V2 X8 @ 3.40 GHz | Intel/Nvidia up 18:28, 2 users, load average: 0.30, 0.86, 0.70
Op dinsdag 12 januari 2021 15:36:14 CET schreef Malcolm:
Hi All That was my understanding as well, move to new platform....
My thought is just archiving off the older posts in the 4.2 release, this becomes read only for users to access (link too etc), newer posts imported into new Forum platform (say 12-18 months)... Same here. And FWIW I'd love to see us move to Discourse.
Also, I don't know where the miscommunication originates. No pointing fingers, not blaming anyone, but I have seen posts here and there about the activity of contributors working on Discourse as a replacement for VB. I agree with Stasiek: I'd rather see us cooperate on that, instead of spending time on multiple options. -- Gertjan Lettink a.k.a. Knurpht openSUSE Board openSUSE Forums Team
Knurpht-openSUSE wrote:
Op dinsdag 12 januari 2021 15:36:14 CET schreef Malcolm:
Hi All That was my understanding as well, move to new platform....
My thought is just archiving off the older posts in the 4.2 release, this becomes read only for users to access (link too etc), newer posts imported into new Forum platform (say 12-18 months)...
Same here. And FWIW I'd love to see us move to Discourse.
Also, I don't know where the miscommunication originates. No pointing fingers, not blaming anyone, but I have seen posts here and there about the activity of contributors working on Discourse as a replacement for VB. I agree with Stasiek: I'd rather see us cooperate on that, instead of spending time on multiple options.
If a move to discourse is imminent, that is great - though in last week's Heroes meeting that was not the impression I got ? In pure practical terms, updating the current vb forums setup would be a Good Thing(r) by enabling the server to run Leap 15.2 and PHP7. Stasiek wrote:
The snag we had, and I can't tell still how important it is, is the thing that also happens with your version of the forums: https://osforums.plexor.ch/forum/deutsch-german/hilfe-und-helfen/installatio... forums before certain date are completely broken due to changes in encoding in php decade ago.
Olav will no doubt chime in later, but the point is that the current forums and the upgrade he has prepared are broken in the same way, except the URL looks better in Olav's version :-) https://osforums.plexor.ch/forum/deutsch-german/hilfe-und-helfen/installatio... https://forums.opensuse.org/showthread.php/444119-verschl%EF%BF%BDsselte-Par... -- Per Jessen, Zürich (1.0°C) Member, openSUSE Heroes
Am Di, 12. Jan, 2021 um 4:48 P. M. schrieb Per Jessen
Olav will no doubt chime in later, but the point is that the current forums and the upgrade he has prepared are broken in the same way, except the URL looks better in Olav's version :-)
https://osforums.plexor.ch/forum/deutsch-german/hilfe-und-helfen/installatio...
https://forums.opensuse.org/showthread.php/444119-verschl%EF%BF%BDsselte-Par...
And I'm not sure if that's a good thing, it's easier to set up the redirects for https://forums.opensuse.org/showthread.php/444119* to the new ids than for titles (that was what I was planning to do with discourse as well) LCP [Stasiek] https://lcp.world
Stasiek Michalski wrote:
Am Di, 12. Jan, 2021 um 4:48 P. M. schrieb Per Jessen
: Olav will no doubt chime in later, but the point is that the current forums and the upgrade he has prepared are broken in the same way, except the URL looks better in Olav's version :-)
https://osforums.plexor.ch/forum/deutsch-german/hilfe-und-helfen/installatio...
https://forums.opensuse.org/showthread.php/444119-verschl%EF%BF%BDsselte-Par...
And I'm not sure if that's a good thing, it's easier to set up the redirects for https://forums.opensuse.org/showthread.php/444119* to the new ids than for titles (that was what I was planning to do with discourse as well)
The URL style is a vBulletin admin option - apparently, the option in Olavs setup is the "mod_rewrite friendly" version :-) https://www.vbulletin.com/docs/html/main/options_seofriendly_urls -- Per Jessen, Zürich (1.2°C) Member, openSUSE Heroes
Hi all, I have been working a bit more on migrating the forums to vBulletin5, and Per and me think it's now in a state good enough to go ahead and carry it out. Among other things, I've looked into styling, and it turns out that it's easy to do. Just by changing the logo, and making it a bit more green, it already looks quite similar to the current production site: https://osforums.plexor.ch/ In my opinion, this level of styling is sufficient, as a first step. However, most of my work during the past weeks has been focused on the character encoding issue with older posts. During that process I came across this post: https://osforums.plexor.ch/forum/english/other-forums/news-announcements/ann... In this post, the change is announced, and the motivation behind it is explained. Interestingly, the forum maintainers at the time knew that the change to Unicode would introduce strange characters in older posts, and accepted it. And urged forum users to accept it, too. Nevertheless, I started looking into creating a tool that can fix the issue, with some success. For example: https://forums.opensuse.org/showthread.php/445384-LVM-Verschl%EF%BF%BDsselt-... vs https://osforums.plexor.ch/forum/deutsch-german/hilfe-und-helfen/installatio... Nice as this may be, it also clearly needs more work. But I'd like to halt that effort for now, and go back to getting the migration to vBulletin5 done first. Per Jessen and me will be performing the migration together, and have sent out a request for a new VM and started doing other preparation work. Regards, Olav On Tue, 2021-01-12 at 16:48 +0100, Per Jessen wrote:
Knurpht-openSUSE wrote:
Op dinsdag 12 januari 2021 15:36:14 CET schreef Malcolm:
Hi All That was my understanding as well, move to new platform....
My thought is just archiving off the older posts in the 4.2 release, this becomes read only for users to access (link too etc), newer posts imported into new Forum platform (say 12-18 months)...
Same here. And FWIW I'd love to see us move to Discourse.
Also, I don't know where the miscommunication originates. No pointing fingers, not blaming anyone, but I have seen posts here and there about the activity of contributors working on Discourse as a replacement for VB. I agree with Stasiek: I'd rather see us cooperate on that, instead of spending time on multiple options.
If a move to discourse is imminent, that is great - though in last week's Heroes meeting that was not the impression I got ?
In pure practical terms, updating the current vb forums setup would be a Good Thing(r) by enabling the server to run Leap 15.2 and PHP7.
Exactly - this is just about moving the current platform forward to a more current software stack while we wait for Discourse or whatever. It's pretty easy to do, and doesn't obstruct any other plans we might have for the forums.
Stasiek wrote:
The snag we had, and I can't tell still how important it is, is the thing that also happens with your version of the forums: https://osforums.plexor.ch/forum/deutsch-german/hilfe-und-helfen/installatio... forums before certain date are completely broken due to changes in encoding in php decade ago.
Olav will no doubt chime in later, but the point is that the current forums and the upgrade he has prepared are broken in the same way, except the URL looks better in Olav's version :-)
I can't take credit for that, of course - it's just the new version of vBulletin doing a better job than the old one. Regards, Olav
On Tue, 2021-01-12 at 08:36 -0600, Malcolm wrote:
My thought is just archiving off the older posts in the 4.2 release, this becomes read only for users to access (link too etc), newer posts imported into new Forum platform (say 12-18 months)...
I think we can do better than that, and I wouldn't like to treat some threads/posts worse than others if it can be avoided. Regards, Olav
On Tue, 2021-01-12 at 15:10 +0100, Stasiek Michalski wrote:
I actually already had a version of our forums running using Discourse, and I think if we are gonna migrate the forums we might as well go with something open source right away ;)
I'm not opposed to that, of course.
The snag we had, and I can't tell still how important it is, is the thing that also happens with your version of the forums: https://osforums.plexor.ch/forum/deutsch-german/hilfe-und-helfen/installatio... forums before certain date are completely broken due to changes in encoding in php decade ago.
I noticed these character encoding errors, too, and decided to ignore them because they also occur in the existing forum. As long as the migration doesn't make anything worse than it currently is, I'm satisfied. Also, fixing the character encoding issues can be solved independently in a separate task, at least if upgrading to vb5. Porting the forums to discourse is probably not as forgiving, though. In that case, it is required to fix the encoding issues first, I guess.
Honestly if you have the time to spend on vb5, I would rather like to see the time spent on migrating to Discourse, and I can probably help in some capacity so we get it all salted as well right away.
I'm not opposed to migrating to Discourse, or helping with it. It's just that I had the impression the effort had petered out a bit, and maybe was semi-abandoned? So my idea was to at least take one small step forward by making sure we at least run on a reasonably modern and supported platform, thereby giving us ample time to work out the next step.
Most important notes in that migration is that you have to use a converter script on the database because if you don't you end up with the only working posts being the ones from before 2011 or so. I used: https://gist.github.com/gschlager/bad57293004c0bfcddba and had success with everything working correctly.
Interesting - I'll give that a go.
Additional point, since discourse maps users onto emails, we have to request SUSE IT to give us a list of usernames matching emails (because vb4 has wrong emails for usernames, I don't think it's important to explain why that is).
Has this been requested already? If not, why?
We agreed with the Forums team on deploying at least https://github.com/sman591/discourse-nntp-bridge. Discourse will have to be set up using OIDC provided by our ipsilon instance at id.opensuse.org, and I hope we can also use the js script that embeds discourse into pages as a comments system for news.opensuse.org, so we have comments functionality there (in the future we may also want to use it for software-o-o, but that's still up in the air).
Sorry, I've been out of the Heroes loop for a while, wasn't aware of that agreement. I do agree that OIDC integration is a must-have, and any other integration between our online assets would also be cool. Regards, Olav
Am Di, 12. Jan, 2021 um 4:53 P. M. schrieb Olav Reinert
Honestly if you have the time to spend on vb5, I would rather like to see the time spent on migrating to Discourse, and I can probably help in some capacity so we get it all salted as well right away.
I'm not opposed to migrating to Discourse, or helping with it. It's just that I had the impression the effort had petered out a bit, and maybe was semi-abandoned?
I'm just doing everything, and don't have much time to do other things. We had a broken matrix telegram bridge for a week just because I couldn't handle that (and also that VM is now hell because of pip, so it's not a fun solution, but it was pretty much the only way to deal with that without leaving people offline for longer).
Additional point, since discourse maps users onto emails, we have to request SUSE IT to give us a list of usernames matching emails (because vb4 has wrong emails for usernames, I don't think it's important to explain why that is).
Has this been requested already? If not, why?
I haven't requested it, before I had the assurance from somebody from SUSE IT we could get it. Bernhard Wiedermann said he would be able to provide it though, so I am counting on that being the case. It has to be done together with the migration because otherwise we would have outdated emails again though. LCP [Stasiek] https://lcp.world
Hello, Am Dienstag, 12. Januar 2021, 15:10:02 CET schrieb Stasiek Michalski:
Most important notes in that migration is that you have to use a converter script on the database because if you don't you end up with the only working posts being the ones from before 2011 or so. I used: https://gist.github.com/gschlager/bad57293004c0bfcddba and had success with everything working correctly.
I looked at the script, and I'm a bit surprised to hear that ;-) (Did you check the charset of old and new posts after running that script?) My understanding of the script is that it changes the charset for whole tables - but sadly we have tables that have (depending on the date) two different charsets. So if the script changes the charset of the whole table, that basically means that one half of the table will be repaired, but the other half will be broken ;-) The script might give some inspiration how we get our charset mix fixed, but we'll most probably need to split the INSERTs at the "charset border", and then insert each half with the correct charset specified. (In theory it's even possible to fix the encoding inside the database, but doing it in a dump is much easier.) Regards, Christian Boltz -- [20:21] <jospoortvliet> ok but IF we do that, note that you'll have to cook for 50+ people. [...] [20:21] <suseROCKs> jospoortvliet, so you're saying you need 50 microwaves??? :-) [from #opensuse-project]
Am Mi, 13. Jan, 2021 um 7:33 P. M. schrieb Christian Boltz
I looked at the script, and I'm a bit surprised to hear that ;-) (Did you check the charset of old and new posts after running that script?)
"Everything working correctly" meaning "Everything is the same as it is currently on forums.opensuse.org"
My understanding of the script is that it changes the charset for whole tables - but sadly we have tables that have (depending on the date) two different charsets. So if the script changes the charset of the whole table, that basically means that one half of the table will be repaired, but the other half will be broken ;-)
That's correct
The script might give some inspiration how we get our charset mix fixed, but we'll most probably need to split the INSERTs at the "charset border", and then insert each half with the correct charset specified.
(In theory it's even possible to fix the encoding inside the database, but doing it in a dump is much easier.)
Hm, that's a pretty cool idea, but working with database dump of this size is a nightmare with text editors and my motivation for that is pretty much nil. Maybe one day when I'm bored enough ;) LCP [Stasiek] https://lcp.world
Stasiek Michalski wrote:
(In theory it's even possible to fix the encoding inside the database, but doing it in a dump is much easier.)
Hm, that's a pretty cool idea, but working with database dump of this size is a nightmare with text editors and my motivation for that is pretty much nil. Maybe one day when I'm bored enough ;)
sed is your friend. -- Per Jessen, Zürich (2.2°C) Member, openSUSE Heroes
participants (6)
-
Christian Boltz
-
Knurpht-openSUSE
-
Malcolm
-
Olav Reinert
-
Per Jessen
-
Stasiek Michalski