Hi, I just would like to follow up the wiki update topic from IRC: This week I would like to start playing with OBS, going to set up an env for it and start looking into the wiki build. If you have any advice throw it my way ;) I have a few pretty hectic weeks at work, but by the end of next week I should be good to work on this project. If all goes well I can keep on maintaining things going forward. If there is a deadline on this please let me know so can try and prioritize things differently. -- Br, A.
Hi Attila ...and welcome! :-) Am Tue, 07 Dec 2021 05:33:55 +0000 schrieb Attila Pinter <adathor@protonmail.com>:
I just would like to follow up the wiki update topic from IRC: This week I would like to start playing with OBS, going to set up an env for it and start looking into the wiki build. If you have any advice throw it my way ;)
To give you something to start playing: https://build.opensuse.org/package/show/home:lrupp:mediawiki/container-openS...
I have a few pretty hectic weeks at work, but by the end of next week I should be good to work on this project. If all goes well I can keep on maintaining things going forward. If there is a deadline on this please let me know so can try and prioritize things differently.
I know that Christian is not that happy with my solution (just taking the mediawiki package from server:php:applications), as it just allows one version of mediawiki, but that should not harm in a fully containerized environment anyway. At the moment, I just see the benefit in us *not* maintaining another application in parallel. (And in case that we really want more than one mediawiki version on one system, I hope we can at least find a temporary solution for any migration.) But I'm happy to discuss this... In case Christian agrees, the next steps would be: 1) https://build.opensuse.org/project/show/openSUSE:infrastructure:wiki -> adjust all plugin packages to build and work successful with the latest mediawiki package. Maybe, we can get rid of one of two of these plugins, but I guess the packaging can happen in parallel to the discussion of the usefulness of each single plugin. 2) Check the search functionality. As Christian said: the most critical point here is elasticsearch. Good: the mediawiki foundation seems to use elasticsearch in their installation as well, so this should see long-term support. Bad: we might need a way to package and run the 'right' elasticsearch version. Note: elasticsearch is currently running on other servers. So we need a dedicated package/containers for this as well. 3) A new VM (at least has host for the containers or for the 'plain packages') to test if everything works. We can use a plain install for this step. Therefor testing can happen everywhere. 4) Check and adjust the Salt setup. Again: testing can happen everywhere... 5) Test migration: use a copy of the database(s) and local files to finally test (and automate) the upgrade of all(!) wikis in one, big step. Note: *all* wikis are currently provided by a single mediawiki installation. Regards, Lars
Hi Lars, ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Wednesday, December 8th, 2021 at 9:39 PM, Lars Vogdt <lars@linux-schulserver.de> wrote:
Hi Attila
...and welcome! :-)
Thank you ::)
Am Tue, 07 Dec 2021 05:33:55 +0000
schrieb Attila Pinter adathor@protonmail.com:
I just would like to follow up the wiki update topic from IRC:
This week I would like to start playing with OBS, going to set up an
env for it and start looking into the wiki build. If you have any
advice throw it my way ;)
To give you something to start playing:
https://build.opensuse.org/package/show/home:lrupp:mediawiki/container-openS...
This is great, I just started to play with OBS (shameful, I know, but better later than never ^_^), and if I got that right we can build from OBS directly to registry-o-o. I think we can set up this project, might need an entrypoint of just shove everything in the Dockerfile and can get it over with fairly quickly.
I have a few pretty hectic weeks at work, but by the end of next week
I should be good to work on this project. If all goes well I can keep
on maintaining things going forward. If there is a deadline on this
please let me know so can try and prioritize things differently.
I know that Christian is not that happy with my solution (just taking
the mediawiki package from server:php:applications), as it just allows
one version of mediawiki, but that should not harm in a fully
containerized environment anyway. At the moment, I just see the benefit
in us not maintaining another application in parallel. (And in case
that we really want more than one mediawiki version on one system, I
hope we can at least find a temporary solution for any migration.) But
I'm happy to discuss this...
I think if we really need to we can host as many different versions as we need to. Don't see the use case why would we, but if we containerize things that will certainly help us in the future. Will also ease updating the application, and OBS can be used to build daily the container so we have no vulnerabilities in the container OS.
In case Christian agrees, the next steps would be:
1. https://build.opensuse.org/project/show/openSUSE:infrastructure:wiki
-> adjust all plugin packages to build and work successful with the
latest mediawiki package. Maybe, we can get rid of one of two of
these plugins, but I guess the packaging can happen in parallel to
the discussion of the usefulness of each single plugin.
2. Check the search functionality. As Christian said: the most critical
point here is elasticsearch. Good: the mediawiki foundation seems to
use elasticsearch in their installation as well, so this should see
long-term support. Bad: we might need a way to package and run the
'right' elasticsearch version.
Note: elasticsearch is currently running on other servers. So we
need a dedicated package/containers for this as well.
As much as I love containers and microservices I would try and avoid containerizing a monolith Java application like Elasticsearch. Christian mentioned this on IRC: "if I get https://www.mediawiki.org/wiki/Extension:CirrusSearch right, we'll need elasticsearch 6.8.x (actually "6.5.4 recommended", but let's try the latest 6.8.x instead of that even older version)". We could package Elasticsearch 6.8/6.5.4, and if memory serves well there are builds available from Elasticsearch to help this process. I think 6.8 will work for us fine, they rarely implement anything major between minor releases.
3. A new VM (at least has host for the containers or for the 'plain
packages') to test if everything works. We can use a plain install
for this step. Therefor testing can happen everywhere.
I plan to set up a local environment as well, but if we go for a new VM could that be on MicroOS? Transactional-updates and SELinux makes it pretty attractive. If it works out - even if we go with Leap - we could just flip the DNS record to point to this new VM in a blue-green deployment fashion?
4. Check and adjust the Salt setup. Again: testing can happen
everywhere...
5. Test migration: use a copy of the database(s) and local files to
finally test (and automate) the upgrade of all(!) wikis in one, big
step.
Note: all wikis are currently provided by a single mediawiki
installation.
Regards,
Lars
Hello, Am Freitag, 10. Dezember 2021, 05:48:58 CET schrieb Attila Pinter:
On Wednesday, December 8th, 2021 at 9:39 PM, Lars Vogdt wrote:
Hi Attila
...and welcome! :-)
Also from me ;-) and sorry for the late answer. [...]
I know that Christian is not that happy with my solution (just taking the mediawiki package from server:php:applications), as it just allows one version of mediawiki, but that should not harm in a fully containerized environment anyway. At the moment, I just see the benefit in us not maintaining another application in parallel. (And in case that we really want more than one mediawiki version on one system, I hope we can at least find a temporary solution for any migration.) But I'm happy to discuss this...
I understand that you don't like to maintain another package, but then - we still have to package and maintain all the extensions (17 packages), so one more package (for MediaWiki itsself) is just a minor annoyance - and IMHO being able to install it in parallel is worth the effort. (I have to admit that I also use the mediawiki_$majorversion packages on my own server, and being able to upgrade one wiki after the other is a requirement for me, so feel free to call this wish selfish ;-) )
I think if we really need to we can host as many different versions as we need to. Don't see the use case why would we, but if we containerize things that will certainly help us in the future.
Having two versions installed in parallel makes upgrades easier, especially if you have 20 wikis on the same server and don't want to upgrade them in one step (with the worst-case-option to see 20 failed upgrades at once ;-)
Will also ease updating the application, and OBS can be used to build daily the container so we have no vulnerabilities in the container OS.
You mean like the "zypper patch" cronjob we have nowadays? ;-)
In case Christian agrees, the next steps would be:
1. https://build.opensuse.org/project/show/openSUSE:infrastructure:wiki -> adjust all plugin packages to build and work successful with the latest mediawiki package. Maybe, we can get rid of one of two of these plugins, but I guess the packaging can happen in parallel to the discussion of the usefulness of each single plugin.
I won't object if we can get rid of some plugins, but as long as they are maintained, finding the one superfluous plugin probably takes longer than just packaging it ;-) (besides that, I don't think that one of the plugins is superfluous - but feel free to correct me) Speaking about packaging the extensions: https://blog.cboltz.de/archives/76-Packaging-MediaWiki-extensions.html describes how I did it last time.
2. Check the search functionality. As Christian said: the most critical> point here is elasticsearch. [...] As much as I love containers and microservices I would try and avoid containerizing a monolith Java application like Elasticsearch.
I'm not a big fan of containers, but I'm still a bit surprised to hear this ;-) What's the reason for avoiding containers in this case?
Christian mentioned this on IRC: "if I get https://www.mediawiki.org/wiki/Extension:CirrusSearch right, we'll need elasticsearch 6.8.x (actually "6.5.4 recommended", but let's try the latest 6.8.x instead of that even older version)". We could package Elasticsearch 6.8/6.5.4, and if memory serves well there are builds available from Elasticsearch to help this process. I think 6.8 will work for us fine, they rarely implement anything major between minor releases.
I also hope so. Maybe we are lucky and can base our package on https://build.opensuse.org/package/show/security:logging/elasticsearch6 (which has the slightly outdated 6.8.12, updating to the latest 6.8.x shouldn't be too hard) and some other packages in security:logging that are needed to build the elasticsearch6 package.
3. A new VM (at least has host for the containers or for the 'plain packages') to test if everything works. We can use a plain install for this step. Therefor testing can happen everywhere.
I plan to set up a local environment as well, but if we go for a new VM could that be on MicroOS? Transactional-updates and SELinux makes it pretty attractive. If it works out - even if we go with Leap - we could just flip the DNS record to point to this new VM in a blue-green deployment fashion?
I'm still not sure if using containers makes sense, but that might be a mix of personal feelings and knowing how much "success" we had with containers in the openSUSE infrastructure so far. Let me ask nevertheless: Do you really think that containers have an advantage compared with a VM when it comes to running the wikis? I could also ask the other way round: we already have a working VM with Apache and PHP running, fully salted, and would "only" need to install the new Mediawiki and switch over some symlinks to it (optionally with salt if we want to switch over all at once). To me, switching to a container sounds like more / additional work. Don't get me wrong - I don't want to discourage you, and even if I'm not a fan of containers (yet?), I'm not completely against using them. However, I'd like to understand if and why using containers would give us advantages (or save us time/work) compared to the existing VM.
4. Check and adjust the Salt setup. Again: testing can happen everywhere...
5. Test migration: use a copy of the database(s) and local files to finally test (and automate) the upgrade of all(!) wikis in one, big step.
I wouldn't recommend to upgrade all wikis at once - in worst case, we end up with 20 broken wikis ;-) Better start with en-test.o.o, then upgrade one of the production wikis, and if nobody starts to scream, we can upgrade the other ones.
Note: all wikis are currently provided by a single mediawiki installation.
Yes and no ;-) They all run on the same VM and have symlinks to the same Mediawiki sources. However, it's easily possible to switch over one wiki/symlink after the other to a new Mediawiki version, which means we can upgrade one wiki after the other. BTW: The upgrade will need some time (database upgrades, maybe also re-creating the search index with the new Elasticsearch), so that alone is a reason to switch over one wiki after the other IMHO. Regards, Christian Boltz PS: random signature, but it somehow fits ;-) -- hey, that's *the cool thing* about software engineering - it's all like a puzzle. And if you ask 10 people about it, then you may get 11 different answers. ;-) [Bernhard Voelker in opensuse-packaging]
Hello, FYI, and to prevent duplicate work: I started the packaging in home:cboltz:infra - packages mediawiki_1_37* (just packaged, but not tested yet) Oh, and I applied the security hotfix from today's Mediawiki security announcement to all wikis. Regards, Christian Boltz -- don't forget autocompletion uses the new iliketomessup framework which is great as it can also be used to transform water into beer. [postdoc38 yahoo fr in https://bugs.kde.org/show_bug.cgi?id=259949]
participants (3)
-
Attila Pinter -
Christian Boltz -
Lars Vogdt