[heroes] Status report
As I will not make it on time for the meeting, maybe not at all - a quick update on the forums migration: Last week we finally managed to get a database export. Part of the delay was really my fault, not enough space on the upload filesystem (although a compressed export would have easily fitted ...). I have also agreed with Jared that he will make a new export whenever needed, so we can have the most current data for the finished migration. I have the newest vBulletin software, it is installed on http://forum.infra.o.o. The database is loaded on the galera cluster (thanks for your help Lars). I used innodb for all tables, I saw no complaints. So far for the good news, now the less good: - I am not yet able to actually run the vBulletin code - currently I get an HTTP 500, which I'm still trying to figure out. - charset. vBulletin recommends is UTF8mb4, but our database is in Latin1. I think(!) this should be simple to convert with ALTER. - stylesheets etc. Apparently this version of vBulletin uses a new setup, not compatible with the current one. I'd like some help here. I'm wondering if some of the current forum admins might want to lend a hand? - authentication. The 'vb_user' table holds 40501 rows. The current system is tied in to the SSO setup in Provo, I don't know how or if this table is used. - according to Marta, the contract with Provo expired on Feb29. MF-IT have agreed to keep running the forums for another week. In summary - if I can solve the HTTP 500 and the database conversion/upgrade works "just like that", the only issue is really authentication. -- Per Jessen, Zürich (6.8°C) Member, openSUSE Heroes -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Hello, Am Montag, 2. März 2020, 13:26:14 CET schrieb Per Jessen:
- I am not yet able to actually run the vBulletin code - currently I get an HTTP 500, which I'm still trying to figure out.
The error_log at least contains a hint: [Mon Mar 02 17:36:48.513389 2020] [php7:error] [pid 17276] [client ...] PHP Fatal error: Uncaught Error: Call to a member function hasAdminPermission() on null in .../htdocs/core/vb/api/wrapper.php:254 Stack trace: #0 .../htdocs/includes/api/interface/collapsed.php(101): vB_Api_Wrapper->__call('fetchCurrentUse...', Array) #1 .../htdocs/includes/vb5/user.php(51): Api_Interface_Collapsed->callApi('user', 'fetchCurrentUse...', Array) #2 .../htdocs/includes/vb5/user.php(40): vB5_User->__construct() #3 .../htdocs/includes/vb5/user.php(75): vB5_User::instance() #4 .../htdocs/includes/vb5/string.php(405): vB5_User::get('lang_charset') #5 .../htdocs/includes/vb5/applicationabstract.php(552): vB5_String::getCharset() #6 .../htdocs/includes/vb5/applicationabstract.php(470): vB5_ApplicationAbstract::setCharset() #7 .../htdocs/includes/vb5/applicationabstract.php(626): vB5_ApplicationAbstract::minErrorPage(false, Object(vB_Exceptio in .../htdocs/core/vb/api/wrapper.php on line 254 I put the server behind the login proxy, and in theory it's now available at https://forums-nbg.opensuse.org/ In practise, it still logs the same error message and gives a 500 :-( I'm sorry that I can't help you more, but maybe you or someone else can make sense from the error message. BTW: Just wondering - are you using a vanilla vBulletin or the one currently running on forums.o.o? At least the favicons are different ;-) Regards, Christian Boltz -- There are no bugs expected after Beta3, that's why it is called RC. [Jan Engelhardt] -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Christian Boltz wrote:
Hello,
Am Montag, 2. März 2020, 13:26:14 CET schrieb Per Jessen:
- I am not yet able to actually run the vBulletin code - currently I get an HTTP 500, which I'm still trying to figure out.
The error_log at least contains a hint:
[Mon Mar 02 17:36:48.513389 2020] [php7:error] [pid 17276] [client [...] PHP Fatal error: Uncaught Error: Call to a member function hasAdminPermission() on null in .../htdocs/core/vb/api/wrapper.php:254
Yep, that's where I am currently at. It could be as simple as a PHP module missing, although I _have_ checked the requirements.
I put the server behind the login proxy, and in theory it's now available at https://forums-nbg.opensuse.org/
Nice, thanks!
BTW: Just wondering - are you using a vanilla vBulletin or the one currently running on forums.o.o? At least the favicons are different ;-)
This is the newest vb5 Connect. -- Per Jessen, Zürich (4.5°C) Member, openSUSE Heroes -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
As I will not make it on time for the meeting, maybe not at all - a quick update on the forums migration:
Last week we finally managed to get a database export. Part of the delay was really my fault, not enough space on the upload filesystem (although a compressed export would have easily fitted ...).
I have also agreed with Jared that he will make a new export whenever needed, so we can have the most current data for the finished migration.
I have the newest vBulletin software, it is installed on http://forum.infra.o.o.
The database is loaded on the galera cluster (thanks for your help Lars). I used innodb for all tables, I saw no complaints.
So far for the good news, now the less good:
- I am not yet able to actually run the vBulletin code - currently I get an HTTP 500, which I'm still trying to figure out.
- charset. vBulletin recommends is UTF8mb4, but our database is in Latin1. I think(!) this should be simple to convert with ALTER.
- stylesheets etc. Apparently this version of vBulletin uses a new setup, not compatible with the current one. I'd like some help here. I'm wondering if some of the current forum admins might want to lend a hand?
- authentication. The 'vb_user' table holds 40501 rows. The current system is tied in to the SSO setup in Provo, I don't know how or if this table is used.
- according to Marta, the contract with Provo expired on Feb29. MF-IT have agreed to keep running the forums for another week.
In summary - if I can solve the HTTP 500 and the database conversion/upgrade works "just like that", the only issue is really authentication. Per, are you using the version the current forums use? You may need that version, then upgrade. The upgrade scripts would make the necessary changes to
Op maandag 2 maart 2020 13:26:14 CET schreef Per Jessen: the DB. Mind, it's years ago for me when I had to take that path, instead of going through all the code and look for DB changes. -- Gertjan Lettink a.k.a. Knurpht openSUSE Forums Team -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Knurpht-openSUSE wrote:
Per, are you using the version the current forums use? You may need that version, then upgrade. The upgrade scripts would make the necessary changes to the DB. Mind, it's years ago for me when I had to take that path, instead of going through all the code and look for DB changes.
Trying to run the old version (4.2 I think) would mean running SLES12, which is only extra work. I am running the newest version, with the expectation it will take a look at the database and say "hey, need to upgrade" and then do it. -- Per Jessen, Zürich (3.9°C) Member, openSUSE Heroes -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Thanks for this update, Per. A couple comments inline below: On Mon, 02 Mar 2020 13:26:14 +0100, Per Jessen wrote:
- I am not yet able to actually run the vBulletin code - currently I get an HTTP 500, which I'm still trying to figure out.
Looking at the error message, I think Gertjan is on track - this is coming from vBulletin 5. I know we talked about the database needing to be upgraded, and that was the plan - but to confirm, did you install the database on version 4 first and then upgrade to version 5?
- authentication. The 'vb_user' table holds 40501 rows. The current system is tied in to the SSO setup in Provo, I don't know how or if this table is used.
The way it works (as I understand it) is that the SSO process sends over data to vBulletin so it can provision the records in the database. vBulletin itself doesn't know about the external authority, so each of those rows of data is a valid user. The passwords that are stored are bogus (as the password exchange takes place over the SSO connection). I don't have a lot of insight into how the SSO process actually works (my testing environment isn't set up for SSO), but typically you need some metadata on your end, and the identity provider (where the user logs in) needs to know a callback URL so it knows where to go after the login is completed. If the callback URL isn't configured properly, SSO won't work - Jared will need to make changes in the MF SSO setup in order to make this work as things stand right now - even suse.com (as of a week ago, at least) still redirects to the MicroFocus login page for authentication. If we opt to not be SSO-enabled, every user is going to need to reset their password manually, and the users will be subject to any security flaws in the vBulletin PHP code (we've had a couple of incidents over the years, but haven't been affected because the vb_user table doesn't store real passwords for anyone other than staff - those passwords are used to access the admin and moderator interfaces). Jim -- Jim Henderson Please keep on-topic replies on the list so everyone benefits -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Jim Henderson wrote:
Thanks for this update, Per. A couple comments inline below:
On Mon, 02 Mar 2020 13:26:14 +0100, Per Jessen wrote:
- I am not yet able to actually run the vBulletin code - currently I get an HTTP 500, which I'm still trying to figure out.
Looking at the error message, I think Gertjan is on track - this is coming from vBulletin 5. I know we talked about the database needing to be upgraded, and that was the plan - but to confirm, did you install the database on version 4 first and then upgrade to version 5?
This is with vb5 connect - I am expecting the code to spot the back-level database and do the upgrade automagically. The idea of running the existing code on the existing database was before we got our hands on the latest version of vB.
- authentication. The 'vb_user' table holds 40501 rows. The current system is tied in to the SSO setup in Provo, I don't know how or if this table is used.
The way it works (as I understand it) is that the SSO process sends over data to vBulletin so it can provision the records in the database. vBulletin itself doesn't know about the external authority, so each of those rows of data is a valid user. The passwords that are stored are bogus
Okay, thanks, good to know. [snip SSO] I am expecting to use the authentication scheme we use for e.g. our wikis and progress.o.o - any reason this would not work? I don't know much about it myself, I'll need some help with that. -- Per Jessen, Zürich (3.9°C) Member, openSUSE Heroes -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
On Tue, 03 Mar 2020 07:25:58 +0100, Per Jessen wrote:
Looking at the error message, I think Gertjan is on track - this is coming from vBulletin 5. I know we talked about the database needing to be upgraded, and that was the plan - but to confirm, did you install the database on version 4 first and then upgrade to version 5?
This is with vb5 connect - I am expecting the code to spot the back-level database and do the upgrade automagically.
vBulletin hasn't traditionally been that smart. There should be an upgrade script in the installation directory IIRC (I haven't looked at vb5, because I don't have access to download it). I do have a Docker container that has a fully-running vBulletin instance in it - I can make that available to you if that would help with validating the database dump you got from Jared.
The idea of running the existing code on the existing database was before we got our hands on the latest version of vB.
I understand - the thing is that vB just doesn't have the brains to automatically upgrade the database. That has to be initiated by the admin.
I am expecting to use the authentication scheme we use for e.g. our wikis and progress.o.o - any reason this would not work? I don't know much about it myself, I'll need some help with that.
I believe we use a SAML 2.0 add-on in our current implementation; logging in through the wiki and progress.o.o, I don't see any SAML traffic from my browser. Kim or Jared could confirm what the plugin we use is for SSO currently. This looks like a MicroFocus Access Manager front-end reverse proxy (the ICSLogin/ICSLogout URIs were originally used as part of the old iChain appliance, but aren't part of the current version of MFAM), which I do remember you mentioning in other messages here. It's possible it's an OAuth 2.0 authentication process that's handled completely by the proxy, and is configured to bypass the typical authorization screen (the one that defines which pieces of data you want to share with the application - that's a typical OAuth 2.0 flow, but it's not unusual to bypass that page, either). I've got a plugin in Chrome to diagnose issues with SSO logins, but it doesn't seem to be working at the moment, so it may well be a SAML connection and I'm just not seeing the traffic for some reason. I don't know the current access manager product very well (I work for a company that makes a competing product), but I know someone who at least used to be active in the openSUSE community who ISTR has pretty good knowledge of it. Let me see if he's amenable to lending a hand. -- Jim Henderson Please keep on-topic replies on the list so everyone benefits -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Jim Henderson wrote:
On Tue, 03 Mar 2020 07:25:58 +0100, Per Jessen wrote:
Looking at the error message, I think Gertjan is on track - this is coming from vBulletin 5. I know we talked about the database needing to be upgraded, and that was the plan - but to confirm, did you install the database on version 4 first and then upgrade to version 5?
This is with vb5 connect - I am expecting the code to spot the back-level database and do the upgrade automagically.
vBulletin hasn't traditionally been that smart. There should be an upgrade script in the installation directory IIRC
Okay, that's a pity, I was expecting something reasonably clever. Still, an upgrade script will do too.
I do have a Docker container that has a fully-running vBulletin instance in it - I can make that available to you if that would help with validating the database dump you got from Jared.
It looks like a plain mysqldump export, and it imported without problems.
The idea of running the existing code on the existing database was before we got our hands on the latest version of vB.
I understand - the thing is that vB just doesn't have the brains to automatically upgrade the database. That has to be initiated by the admin.
Right - I'll have to go do some more reading.
I am expecting to use the authentication scheme we use for e.g. our wikis and progress.o.o - any reason this would not work? I don't know much about it myself, I'll need some help with that.
I believe we use a SAML 2.0 add-on in our current implementation; logging in through the wiki and progress.o.o, I don't see any SAML traffic from my browser. Kim or Jared could confirm what the plugin we use is for SSO currently.
I guess I could have a look myself, Kim sent me a tarball with everything. -- Per Jessen, Zürich (3.9°C) Member, openSUSE Heroes -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
On Tue, 03 Mar 2020 18:45:54 +0100, Per Jessen wrote:
I do have a Docker container that has a fully-running vBulletin instance in it - I can make that available to you if that would help with validating the database dump you got from Jared.
It looks like a plain mysqldump export, and it imported without problems.
I know that MF felt it necessary to scrub some e-mail addresses (their internal folks' addresses) from the user table. That *shouldn't* cause a problem (until those users try to log in and the account isn't recognized because the address was scrubbed, and that's what links the vBulletin user account to the SSO account, as I recall).
I am expecting to use the authentication scheme we use for e.g. our wikis and progress.o.o - any reason this would not work? I don't know much about it myself, I'll need some help with that.
I believe we use a SAML 2.0 add-on in our current implementation; logging in through the wiki and progress.o.o, I don't see any SAML traffic from my browser. Kim or Jared could confirm what the plugin we use is for SSO currently.
I guess I could have a look myself, Kim sent me a tarball with everything.
If it's the same tarball I got, I don't think the SAML plugin was included. All of that stuff is locked behind a login on the vBulletin site, I believe, so only the person with login credentials to their customer forum/site would be able to grab those plugins. If you want the Docker container, let me know - I'll put it somewhere where you can grab it. It's based on the official SLES 12 docker image. I also have a VirtualBox VM that I had previously used for testing that I could export as an OVF and make available. Jim -- Jim Henderson Please keep on-topic replies on the list so everyone benefits -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Jim Henderson wrote:
On Tue, 03 Mar 2020 18:45:54 +0100, Per Jessen wrote:
I do have a Docker container that has a fully-running vBulletin instance in it - I can make that available to you if that would help with validating the database dump you got from Jared.
It looks like a plain mysqldump export, and it imported without problems.
I know that MF felt it necessary to scrub some e-mail addresses (their internal folks' addresses) from the user table.
Yep, that was discussed, and I see no 'microfocus' addresses in the table. No blank addresses either.
I guess I could have a look myself, Kim sent me a tarball with everything.
If it's the same tarball I got, I don't think the SAML plugin was included. All of that stuff is locked behind a login on the vBulletin site, I believe, so only the person with login credentials to their customer forum/site would be able to grab those plugins.
If you want the Docker container, let me know - I'll put it somewhere where you can grab it. It's based on the official SLES 12 docker image. I also have a VirtualBox VM that I had previously used for testing that I could export as an OVF and make available.
Thanks Jim - your hint with the upgrade script already helped me go past the http500. -- Per Jessen, Zürich (3.5°C) Member, openSUSE Heroes -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
On Tue, 03 Mar 2020 20:01:51 +0100, Per Jessen wrote:
I know that MF felt it necessary to scrub some e-mail addresses (their internal folks' addresses) from the user table.
Yep, that was discussed, and I see no 'microfocus' addresses in the table. No blank addresses either.
Sounds good. In the database dump, is there a way to identify the users who had their e-mail address modified? I'm thinking that if we need to 'reconnect' the user to their account down the road, it would be useful if there was a way to see how many people were affected.
Thanks Jim - your hint with the upgrade script already helped me go past the http500.
Excellent. Let me know if I can help with anything else - and I'll let you know when I hear back from the person I'm asking about the SSO side of things. -- Jim Henderson Please keep on-topic replies on the list so everyone benefits -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
Jim Henderson wrote:
On Tue, 03 Mar 2020 20:01:51 +0100, Per Jessen wrote:
I know that MF felt it necessary to scrub some e-mail addresses (their internal folks' addresses) from the user table.
Yep, that was discussed, and I see no 'microfocus' addresses in the table. No blank addresses either.
Sounds good. In the database dump, is there a way to identify the users who had their e-mail address modified? I'm thinking that if we need to 'reconnect' the user to their account down the road, it would be useful if there was a way to see how many people were affected.
I'll check to see if there is an update timestamp. Please remind me if I forget.
Thanks Jim - your hint with the upgrade script already helped me go past the http500.
Excellent. Let me know if I can help with anything else - and I'll let you know when I hear back from the person I'm asking about the SSO side of things.
We have people here too who will be more knowledgeable than me, I'm just really not aware of how it works today. Any and all help appreciated. -- Per Jessen, Zürich (2.8°C) Member, openSUSE Heroes -- To unsubscribe, e-mail: heroes+unsubscribe@opensuse.org To contact the owner, e-mail: heroes+owner@opensuse.org
participants (4)
-
Christian Boltz -
Jim Henderson -
Knurpht-openSUSE -
Per Jessen