October 2017 - Heroes - openSUSE Mailing Lists

[heroes] Status Galera Cluster | syslog server | ntp
by Lars Vogdt 14 Nov '17

14 Nov '17

Hi folks --- galera I've some good news: the new Galera cluster for our MySQL databases is up and running. I've enabled monitoring and the (db)proxy setup already, but did not start salt on the machines, yet. During the next days, there should be some time to do some testing and finally start with the import of the current databases one by one. One by one because I want to check first if really all the databases in the old cluster are still needed - IMHO a lot of them can simply be skipped. I like to start with the redmine (aka progress) DB first - and then decide about the rest - objections? ...and of course: any help welcome! Especially from those who have experience with galera clusters or mysql database setups/tuning. --- syslog A note to our Salt gurus: syslog.infra.opensuse.org is up and running, but does not see a lot of incoming events, yet. Can you please place something like: ---- /etc/rsyslog.d/remote.conf $WorkDirectory /var/spool/rsyslog $ActionQueueFileName uniqName $ActionQueueMaxDiskSpace 1g $ActionQueueSaveOnShutdown on $ActionQueueType LinkedList $ActionResumeRetryCount -1 *.* @@syslog.infra.opensuse.org ----- On all our nodes together with an entry like: ----- /etc/hosts 192.168.47.7 syslog.infra.opensuse.org syslog monitor ----- for the case that DNS does not work or the name is not resolvable during boot. --- ntp First of all: I switched to use only ntp1, ntp2 and ntp3 on the nodes. This way, the first DNS entry wins, which is lucky most of the time ntp1.infra.opensuse.org, ntp2.infra.opensuse.org and ntp3.infra.opensuse.org. I did not adapt the salt profile, yet, because I want to hear from you if we might switch over from ntp to chrony ? --- CU ...and have a good night ;-) Regards, Lars -- To unsubscribe, e-mail: heroes+unsubscribe(a)opensuse.org To contact the owner, e-mail: heroes+owner(a)opensuse.org

2 3

[heroes] unbound problem - rfc1918 reverse lookups
by Per Jessen 28 Oct '17

28 Oct '17

Grüezi mitenand I'm having a minor issue with my unbound setup (for directing lookups of infra.o.o over the vpn). Normally reverse lookups would just go straight to our core nameserver, but: # host 192.168.2.159 Host 159.2.168.192.in-addr.arpa not found: 3(NXDOMAIN) It looks like reverse lookups are not being forwarded as per the config. Apparently unbound has default setups for the RFC1918 ranges (AS112), so I am supposed to add: local-zone: "in-addr.arpa." transparent (either nodefault or transparent). It doesn't seem to be working though. Does anyone have something like this working? This is my config: server: local-zone: "in-addr.arpa." nodefault local-zone: "infra.opensuse.org." nodefault stub-zone: name: "infra.opensuse.org" stub-addr: 192.168.254.101 stub-addr: 192.168.254.102 forward-zone: name: "." forward-addr: 192.168.2.254 forward-addr: 2a03:7520:4c68:1::1000 -- Per Jessen, Zürich (9.9°C) openSUSE mailing list admin -- To unsubscribe, e-mail: heroes+unsubscribe(a)opensuse.org To contact the owner, e-mail: heroes+owner(a)opensuse.org

3 4

[heroes] Something changed in mirroring?
by Per Jessen 25 Oct '17

25 Oct '17

After Lars fixed the setup over the weekend, traffic on our mirror (mirror.hostsuisse.com)) resumed the normal levels. Yesterday however, traffic was almost four times more than normal: +------------+-------------------+ | date(ts) | volume(Mb) | +------------+-------------------+ | 2017-10-14 | 7362.7095 | | 2017-10-15 | 3770.8519 | | 2017-10-16 | 2054.8116 | | 2017-10-17 | 1849.7966 | | 2017-10-18 | 11440.7487 | | 2017-10-19 | 6635.4341 | | 2017-10-20 | 6314.6200 | | 2017-10-21 | 59754.2535 | | 2017-10-22 | 60727.1530 | | 2017-10-23 | 58692.0817 | | 2017-10-24 | 201775.5079 | +------------+-------------------+ Normally our mix of clients looks like this: +---------+---------+ | country | clients | +---------+---------+ | it | 187 | | ch | 154 | | es | 118 | | fr | 99 | | br | 29 | | be | 28 | | nl | 16 | | de | 15 | Yesterday: +---------+---------+ | country | clients | +---------+---------+ | de | 669 | | us | 297 | | it | 268 | | es | 187 | | ch | 169 | | fr | 137 | | uk | 127 | | nl | 99 | | pl | 66 | It is most unusual to see so many US and DE clients. Plenty of good mirrors in Germany. -- Per Jessen, Zürich (11.0°C) openSUSE mailing list admin -- To unsubscribe, e-mail: heroes+unsubscribe(a)opensuse.org To contact the owner, e-mail: heroes+owner(a)opensuse.org

2 2

Re: [heroes] much lower mirror activity since Friday?
by Per Jessen 23 Oct '17

23 Oct '17

Levon Avagyan wrote: > Dear Per, > > For sure rsync is not related. > Problem is clearly with mirror brain. > Please have a look at attached picture. Only option is to download directly. Hi Levon that is weird - I have just tried the same, it looks like it's a problem on IPv4. With IPv6, it worked fine. I'm cc'ing mirror(a)o.o. On Levon's screenshot, it is clear that no mirrors were found for: http://download.opensuse.org/distribution/leap/42.2/iso/openSUSE-Leap-42.2-… When I tried from ipv6, I got two mirrors, but when I tried from an IPv4 in Germany, I also got no mirrors. -- Per Jessen, Zürich (18.1°C) openSUSE mailing list admin -- To unsubscribe, e-mail: heroes+unsubscribe(a)opensuse.org To contact the owner, e-mail: heroes+owner(a)opensuse.org

3 3

[heroes] Mirrorbrain broken
by Richard Brown 21 Oct '17

21 Oct '17

Hi Lars, Heroes Terrible download speeds, lots of complaints from users, mirrors on mirror@ reporting less traffic than usual Please investigate -- To unsubscribe, e-mail: heroes+unsubscribe(a)opensuse.org To contact the owner, e-mail: heroes+owner(a)opensuse.org

3 3

[heroes] login.microfocus.com access delay waiting for/transferring data from www.google.com...
by Felix Miata 19 Oct '17

19 Oct '17

Why does logging into BOO involve Google? -- "Wisdom is supreme; therefore get wisdom. Whatever else you get, get wisdom." Proverbs 4:7 (New Living Translation) Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/ -- To unsubscribe, e-mail: heroes+unsubscribe(a)opensuse.org To contact the owner, e-mail: heroes+owner(a)opensuse.org

1 0

Re: [heroes] much lower mirror activity since Friday?
by Per Jessen 19 Oct '17

19 Oct '17

Levon Avagyan wrote: > Hello, > > Same situation with our mirror, just a fraction of traffic volume. > > https://opensuse.ucom.am/ > > Best regards > Levon Avagyan Thanks for your report. I'm still keeping en eye on ourt mirror - we got the sync to work again yesterday, but I don't really see how it would have affected the mirror traffic. -- Per Jessen, Zürich (16.2°C) openSUSE mailing list admin -- To unsubscribe, e-mail: heroes+unsubscribe(a)opensuse.org To contact the owner, e-mail: heroes+owner(a)opensuse.org

1 0

[heroes] much lower mirror activity since Friday?
by Per Jessen 18 Oct '17

18 Oct '17

Since Friday last week, I'm seeing much reduced activity on our mirror (http://mirror.hostsuisse.com) iIt looks like a drop by at least a faktor 10: (this is only ZYpp traffic): For tumbleweed: | 2017-10-12 | 209220 | | 2017-10-13 | 151444 | | 2017-10-14 | 2162 | | 2017-10-15 | 11140 | | 2017-10-16 | 8788 | | 2017-10-17 | 6248 | For distribution/leap/42.3: | 2017-10-12 | 56170 | | 2017-10-13 | 65722 | | 2017-10-14 | 31433 | | 2017-10-15 | 2963 | | 2017-10-16 | 583 | | 2017-10-17 | 2243 | For distribution/leap/42.2: | 2017-10-12 | 7568 | | 2017-10-13 | 9328 | | 2017-10-14 | 4398 | | 2017-10-15 | 70 | | 2017-10-16 | 170 | | 2017-10-17 | 168 | I guess it could it be normal, but it is an odd coincidence. -- Per Jessen, Zürich (17.8°C) openSUSE mailing list admin -- To unsubscribe, e-mail: heroes+unsubscribe(a)opensuse.org To contact the owner, e-mail: heroes+owner(a)opensuse.org

1 1

[heroes] keyserver.o.o - a love story
by Thorsten Bro 17 Oct '17

17 Oct '17

Hi all, like in every good and trashy hollywood/bollywood love movie, two individuums - in this case the keyserver and me - want to exchange their data. But of course - as always in those movies - a bad situation comes up, the dad of the keyserver is against our data exchange and everything ends up in a mess... Maybe - the dad of the keyserver - reconsiders his choice and gives me the possibility to upload keys and sync the keyserver with the outside world again, as I try to work with the keyserver every day, I promise to always upload there an updated key of myself and really promise to be a valuable part of the GPG community signing only keys where I checked a passport and swear on the book of "Honest Ahmed Used Cars and Certificates - Keysigning howto" before. Furthermore, I will consider everybody to sign keys, as it makes the community more valuable. So please dad of keyserver.o.o - give yourself a small kick - and bring this nice service back2life - which means - let it sync with the outside world - pool it with sks - and let the GPG pubkeys have many many beautiful copies out there in the world which will make the family bigger :). Best regards, a sad user -- Thorsten -- To unsubscribe, e-mail: heroes+unsubscribe(a)opensuse.org To contact the owner, e-mail: heroes+owner(a)opensuse.org

1 0

[heroes] More power outage fun
by Christian Boltz 13 Oct '17

13 Oct '17

Hello, as you might have noticed on IRC, Theo and I had some fun to get salt running on riesling2 and water2. In the end, we set up a saltmaster on water2 using local git checkouts (uploaded as tarball) instead of gitfs - it turned out the git connection to gitlab was too slow for whatever reason [1], and sometimes even git clone timed out :-/ I had already copied over all files and database dumps and setup MySQL, and we now have a copy of all wikis running in Provo :-) [2] Note that the wikis are in read-only mode - I want to avoid that I have to sync back changes to Nuremberg ;-) Also, edits done today are missing, but they'll of course be back after switching back to the VMs in Nuremberg. The only missing part is the haproxy setup for all wikis - since I'm not familiar with that, someone else has to do this ;-) Regards, Christian Boltz [1] Theo's guess is that the VPN tunnel is too slow [2] once salt was working, this was the easy part - salt highstate (+ some /etc/hosts tricks) did all the wiki setup :-) -- > ...womit die Geschichte wieder von vorn losgeht... Jaha, aber nun ist das eine vollkommen andere Situation: Jetzt hast Du nämlich eine von Suse generierte kdmrc für ein von Suse geliefertes KDE, und KDE ist eine vom Installationssupport abgedeckte Komponente. Also ist das ein meldefähiger und supportberechtigter Bug! :-) [> Gerald Martin und Kristian Köhntopp in suse-linux] -- To unsubscribe, e-mail: heroes+unsubscribe(a)opensuse.org To contact the owner, e-mail: heroes+owner(a)opensuse.org

2 2

[heroes] Status power outage preparation
by Lars Vogdt 12 Oct '17

12 Oct '17

Hi Just want to give a short overview about the things I was able to do so far: 1) provo-mirror.opensuse.org * The machine is up and running, currently as a normal mirror server. * The mirrorbrain frontend is prepared and I took the freedom to play along with the mod_autoindex_mb to provide a bit more information to our mirror users: https://provo-mirror.opensuse.org/ => the content is in the mirrorbrain-icons RPM in openSUSE:infrastructure - submit requests welcome ;-) * https://mirrors.opensuse.org will also be handled by this machine during the downtime 2) new mirrordb machine * There is a new mirrordb machine named nala running in Provo inside a private network (same as in Nuremberg) - this mirrordb machine will just have a copy (dump) of the Nuremberg database. No updates aka scanner is planned to run during the downtime. 3) new status machine * status2.opensuse.org already has taken over the alias status.opensuse.org from status1.opensuse.org * there is a MySQL master - master setup between the two nodes (SSL encrypted, of course, but I have 250ms latency...). So far everything seems to work, thanks to the low amount of changes on status.opensuse.org 4) new keyserver machine * https://keyserver2.opensuse.org works already * I need to wait for someone who can open the needed additional ports in the firewall to get the commandline "gpg" calls also work * in general, it should be fine to point the keyserver alias ot the new machine during the downtime 5) new login machine * apache and ldap are ready * the ldap proxy can not reach the master server - this is clearly something to investigate after the downtime. I need to find a way to connect into the DMZ like on the other login nodes... That's it so far. For tomorrow, the major part will be the switch over of download.opensuse.org to the provo-mirror... CU, Lars -- To unsubscribe, e-mail: heroes+unsubscribe(a)opensuse.org To contact the owner, e-mail: heroes+owner(a)opensuse.org

2 1

[heroes] changing the TTL of the opensuse.org
by Theo Chatzimichos 12 Oct '17

12 Oct '17

Hello, we need to request from MF-IT to change the TTL of the opensuse.org zone. I would suggest to change it to 4 days, and the change should be done tomorrow around 16:00. Before the change is done, we will need to switch our DNS to point to the provo haproxy. Does that sound fine? If so, I'll file a ticket with MF-IT today -- Theo Chatzimichos <tampakrap(a)opensuse.org> <tchatzimichos(a)suse.com> System Administrator SUSE Operations and Services Team

3 5

[heroes] new temp mx servers
by Theo Chatzimichos 12 Oct '17

12 Oct '17

Hello, mcaj darix and I set up two new MX servers at the Prague SUSE office, that will be working only for the upcoming outage. We have just added them to the MX records of the opensuse.org zone with lower priority. Please let me know of any issues you may encounter Theo

1 0

[heroes] shop.opensuse.org
by ddemaio 10 Oct '17

10 Oct '17

Hi all, I wanted to see if anyone here can help with shop.o.o. to put the new openSUSE Powered Infinity Laptop by Tuxedo Computers on the shop once the board finalizes an agreement with them. In the layout, we would want to have areas to grow to add other laptops like system76, slimbook.es and others should they choose to make a laptop that we can justify marketing on shop.o.o. Could you please let me know if you can help. v/r Doug

1 0

[heroes] renumbering our VMs
by Theo Chatzimichos 04 Oct '17

04 Oct '17

Hello, jfyi I have started today renumbering our VMs from 192.168.254.0/24 to 192.168.47.0/24. Some core services (eg LDAP, proxy, DB, gateway) have both IPs still, so that I don't break public services. Please let me know of any issues you may encounter. -- Theo Chatzimichos <tampakrap(a)opensuse.org> <tchatzimichos(a)suse.com> System Administrator SUSE Operations and Services Team

3 4

[heroes] Re: LAS website
by ddemaio 04 Oct '17

04 Oct '17

Hi Sri, I'm ccing the OSEM team and our Heroes team for infra. I'm not certain if we have a server to host the LAS.GNOME use. The Heroes team would have a better idea about this. You can find the code on https://github.com/openSUSE/osem and will see that it has many contributions from various other projects. v/r Doug On 10/01/2017 12:07 AM, Sriram Ramkrishna wrote: > Just to be clear, the ask is if you would be able to create a server > and have OSEM running on it. For some unfortunate reason, we cannot > host it inside our own infrastructure. > > sri > > On Sat, Sep 30, 2017 at 4:05 PM Sriram Ramkrishna <sri(a)ramkrishna.me > <mailto:sri@ramkrishna.me>> wrote: > > Doug, > > I think this year, we want to move to the events website OSEM. > How do we go about doing this? While the other software was > adequate, it had a lot of design issues and I also it might be > worth going to a software that has solid support, eg more than one > overworked person. :-) > > Thanks! > sri >

3 2

[heroes] FYI: monitoring sends out notifications to #opensuse-admin channel
by Lars Vogdt 03 Oct '17

03 Oct '17

$ubject says it all: I installed a new bot "heroes-bot" in the #opensuse-admin channel on freenode that will get notifications for all monitored machines. I hope the provided information is useful - otherwise please speak up, so we can change it. with kind regards, Lars -- To unsubscribe, e-mail: heroes+unsubscribe(a)opensuse.org To contact the owner, e-mail: heroes+owner(a)opensuse.org

3 4

[heroes] en.opensuse.org - bad webmaster email address
by Sebastian 03 Oct '17

03 Oct '17

Dear admins, Maybe you already know that the english wiki is currently offline (Bad gateway - DNS lookup failure for: proxy-opensuse-ha.login.infra.opensuse.org) but could you please set a valid email address as webmaster at the webserver? It currently says > If you think this is a server error, please contact the webmaster. While webmaster links to > mailto:[no address given] Thanks! Sebastian -- python programming - mail server - photo - video - https://sebix.at cryptographic key at https://sebix.at/DC9B463B.asc and on public keyservers

1 0

[heroes] Reminder: Heroes meeting on Tuesday
by Christian Boltz 02 Oct '17

02 Oct '17

Hello, the next Heroes meeting is planned on Tuesday 2017-10-03 18:00 UTC / 20:00 CEST in #opensuse-admin. See https://progress.opensuse.org/issues/25718 for the topics, and feel free to add other topics we should discuss. Since we skipped the September meeting [1], I copied over all copies to the October meeting. So far, the only additional topic is the planned power outage. Regards, Christian Boltz [1] On the usual date, not too many people were online and we decided to move the meeting by a week. The week later, even less people were there. -- we now have in Staging: [...] * Perl 5.24.0 - It would not be perl if nothing would break [Dominique Leuenberger in opensuse-factory] -- To unsubscribe, e-mail: heroes+unsubscribe(a)opensuse.org To contact the owner, e-mail: heroes+owner(a)opensuse.org

2 1

[heroes] mirror.hostsuisse.com - dropped again?
by Per Jessen 02 Oct '17

02 Oct '17

It looks like our mirror (mirror.hostsuisse.com) was "dropped" again - yesterday at 11:50. Today at 14:07 I ran "mb scan mirror.hostsuisse.com" on olaf, which virtually immediately restarted the mirror traffic. Nothing was touched on our side. Even when the mirror is "back", it is only marked as serving: 42.3 iso 43.2 repo 42.2 iso 42.2 repo But we also mirror TW, 42.x source+updates and ports/. This situation is not new, I've seen that for a while, and I just cannot figure why. I looked at many of the other mirrors - on olaf, I ran "mb list -N" which produced a list of 212 mirrors, of which 179 with nfiles=0 ? (/tmp/mb-list.txt). Looking in my logs since mid-September, I don't see a single access of "update/leap/42.3/oss/repodata/repomd.xml", the marker for "42.3 Updates". Something is not right? -- Per Jessen, Zürich (16.0°C) openSUSE mailing list admin -- To unsubscribe, e-mail: heroes+unsubscribe(a)opensuse.org To contact the owner, e-mail: heroes+owner(a)opensuse.org

1 0