[opensuse-buildservice] broken builds due to "interconnect error: rpc timeout" with PMBS
Hi, I stumbled across broken builds in a local OBS due to "interconnect error: rpc timeout" for packages, that link to PMBS. PMBS is setup as: <project name="Packman"> <title>Packman Build Service PMBS</title> <description>This instance can be used to access resources from Packman.</ description> <remoteurl>https://pmbs-api.links2linux.de/public</remoteurl> </project> OBS is fully updated from OBS:/Server:/2.10/openSUSE_15.1. The magic spell, that comes closest to mitigate the issue *ought* to be: obs_admin --recheck-dod https://pmbs-api.links2linux.de or obs_admin --recheck-dod https://pmbs-api.links2linux.de/public but even after creating the missing /srv/obs/dods directory, things still misbehave. Is this a PMBS issue, a local configuration issue, or something completely different?!? Thanks in advance, Pete -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Am Montag, 23. Dezember 2019, 10:19:53 CET schrieb Hans-Peter Jansen:
Hi,
I stumbled across broken builds in a local OBS due to "interconnect error: rpc timeout" for packages, that link to PMBS.
PMBS is setup as:
<project name="Packman"> <title>Packman Build Service PMBS</title> <description>This instance can be used to access resources from Packman.</ description> <remoteurl>https://pmbs-api.links2linux.de/public</remoteurl> </project>
OBS is fully updated from OBS:/Server:/2.10/openSUSE_15.1.
The magic spell, that comes closest to mitigate the issue *ought* to be:
obs_admin --recheck-dod https://pmbs-api.links2linux.de
or
obs_admin --recheck-dod https://pmbs-api.links2linux.de/public
but even after creating the missing /srv/obs/dods directory, things still misbehave.
Is this a PMBS issue, a local configuration issue, or something completely different?!?
Anything wrong with this question? Any help solving this issue is much appreciated. Thanks, Pete -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Wed, Dec 25, 2019 at 12:37 PM Hans-Peter Jansen <hpj@urpla.net> wrote:
Am Montag, 23. Dezember 2019, 10:19:53 CET schrieb Hans-Peter Jansen:
Hi,
I stumbled across broken builds in a local OBS due to "interconnect error: rpc timeout" for packages, that link to PMBS.
PMBS is setup as:
<project name="Packman"> <title>Packman Build Service PMBS</title> <description>This instance can be used to access resources from Packman.</ description> <remoteurl>https://pmbs-api.links2linux.de/public</remoteurl> </project>
OBS is fully updated from OBS:/Server:/2.10/openSUSE_15.1.
The magic spell, that comes closest to mitigate the issue *ought* to be:
obs_admin --recheck-dod https://pmbs-api.links2linux.de
or
obs_admin --recheck-dod https://pmbs-api.links2linux.de/public
but even after creating the missing /srv/obs/dods directory, things still misbehave.
Is this a PMBS issue, a local configuration issue, or something completely different?!?
Anything wrong with this question?
This is a PMBS issue. The API endpoint for PMBS is probably misbehaving. -- 真実はいつも一つ！/ Always, there's only one truth! -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Am Mittwoch, 25. Dezember 2019, 18:40:51 CET schrieb Neal Gompa:
On Wed, Dec 25, 2019 at 12:37 PM Hans-Peter Jansen <hpj@urpla.net> wrote:
Am Montag, 23. Dezember 2019, 10:19:53 CET schrieb Hans-Peter Jansen:
Hi,
I stumbled across broken builds in a local OBS due to "interconnect error: rpc timeout" for packages, that link to PMBS.
PMBS is setup as:
<project name="Packman">
<title>Packman Build Service PMBS</title> <description>This instance can be used to access resources from Packman.</
description>
<remoteurl>https://pmbs-api.links2linux.de/public</remoteurl>
</project>
OBS is fully updated from OBS:/Server:/2.10/openSUSE_15.1.
The magic spell, that comes closest to mitigate the issue *ought* to be: obs_admin --recheck-dod https://pmbs-api.links2linux.de
or
obs_admin --recheck-dod https://pmbs-api.links2linux.de/public
but even after creating the missing /srv/obs/dods directory, things still misbehave.
Is this a PMBS issue, a local configuration issue, or something completely different?!?
Anything wrong with this question?
This is a PMBS issue. The API endpoint for PMBS is probably misbehaving.
Thanks Neal, for your answer, but simply fetching the public URL looks pretty similar, compared to openSUSE.org: # openSUSE.org $ wget https://api.opensuse.org/public --2019-12-26 18:47:45-- https://api.opensuse.org/public Auflösen des Hostnamens api.opensuse.org (api.opensuse.org)… 195.135.221.133, 2620:113:80c0:8::12 Verbindungsaufbau zu api.opensuse.org (api.opensuse.org)|195.135.221.133|:443 … verbunden. HTTP-Anforderung gesendet, auf Antwort wird gewartet … 302 Found Platz: https://api.opensuse.org/about [folgend] --2019-12-26 18:47:45-- https://api.opensuse.org/about Wiederverwendung der bestehenden Verbindung zu api.opensuse.org:443. HTTP-Anforderung gesendet, auf Antwort wird gewartet … 200 OK Länge: nicht spezifiziert [application/xml] Wird in »public« gespeichert. public [ <=> ] 300 --.-KB/s in 0s 2019-12-26 18:47:45 (13,3 MB/s) - »public« gespeichert [300] $ cat public <about> <title>Open Build Service API</title> <description>API to the Open Build Service</description> <revision>2.11~alpha.20191218T101855.4c27b561b8</revision> <last_deployment>2019-12-18 10:12:06 +0000</last_deployment> <commit>4c27b561b88c09a29babc95f0cdcaf72046876c2</commit> </about> # PMBS $ wget https://pmbs-api.links2linux.de/public --2019-12-26 18:48:08-- https://pmbs-api.links2linux.de/public Auflösen des Hostnamens pmbs-api.links2linux.de (pmbs-api.links2linux.de)… 95.216.108.108 Verbindungsaufbau zu pmbs-api.links2linux.de (pmbs-api.links2linux.de)| 95.216.108.108|:443 … verbunden. HTTP-Anforderung gesendet, auf Antwort wird gewartet … 302 Found Platz: https://pmbs-api.links2linux.de/about [folgend] --2019-12-26 18:48:08-- https://pmbs-api.links2linux.de/about Wiederverwendung der bestehenden Verbindung zu pmbs-api.links2linux.de:443. HTTP-Anforderung gesendet, auf Antwort wird gewartet … 200 OK Länge: nicht spezifiziert [application/xml] Wird in »public.1« gespeichert. public.1 [ <=> ] 267 --.-KB/s in 0s 2019-12-26 18:48:08 (22,5 MB/s) - »public.1« gespeichert [267] $ cat public.1 <about> <title>Open Build Service API</title> <description>API to the Open Build Service</description> <revision>2.10</revision> <last_deployment>2019-07-05 14:35:37 +0200</last_deployment> <commit>1b66a6c499ef52f61406c333d92731323d852fbc</commit> </about> Sure, this doesn't tell anything about specific API access to specific packages. /srv/obs/log/scheduler_x86_64.log shows this: 2019-12-26 18:07:17: [2762] waiting for an event... response from watcher for https://pmbs-api.links2linux.de/public 400 remote error: document element must be 'events', was 'html' retrying in 60 seconds Cheers, Pete -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Thursday 2019-12-26 19:13, Hans-Peter Jansen wrote:
/srv/obs/log/scheduler_x86_64.log shows this:
2019-12-26 18:07:17: [2762] waiting for an event... response from watcher for https://pmbs-api.links2linux.de/public 400 remote error: document element must be 'events', was 'html' retrying in 60 seconds
HTML error page of some kind (i.e. of the "readable" kind that you could open with your browser too, not some XML that needs a special parser). opensuse.org no longer seems to like the request that PMBS made. Hooray for API stability, hooray for... not seeing any of the HTTP headers sent or received :-/ -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Hi Jan, Am Donnerstag, 26. Dezember 2019, 21:16:04 CET schrieb Jan Engelhardt:
On Thursday 2019-12-26 19:13, Hans-Peter Jansen wrote:
/srv/obs/log/scheduler_x86_64.log shows this:
2019-12-26 18:07:17: [2762] waiting for an event... response from watcher for https://pmbs-api.links2linux.de/public 400 remote error: document element must be 'events', was 'html' retrying in 60 seconds
HTML error page of some kind (i.e. of the "readable" kind that you could open with your browser too, not some XML that needs a special parser). opensuse.org no longer seems to like the request that PMBS made. Hooray for API stability, hooray for... not seeing any of the HTTP headers sent or received :-/
Another interesting observation: it indeed makes a difference, if I access the public path of OBS and PMBS with curl or wget. (All done from the local obs server): $ wget -qO- https://pmbs-api.links2linux.de/public <about> <title>Open Build Service API</title> <description>API to the Open Build Service</description> <revision>2.10</revision> <last_deployment>2019-07-05 14:35:37 +0200</last_deployment> <commit>1b66a6c499ef52f61406c333d92731323d852fbc</commit> </about> $ curl -o- https://pmbs-api.links2linux.de/public <html><body>You are being <a href="https://pmbs.links2linux.de/ about">redirected</a>.</body></html> but the behavior is consistent with OBS. I can provide many more details, but still have no idea, where these issues stem from. It just shows any package build as broken, that links to PMBS. I don't think, it's API related, it smells more like some transient issue, that eventually turned chronic.. Cheers, Pete -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
On Dez 27 2019, Hans-Peter Jansen wrote:
$ curl -o- https://pmbs-api.links2linux.de/public
You need curl -L. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
Am Freitag, 27. Dezember 2019, 23:02:13 CET schrieb Andreas Schwab:
On Dez 27 2019, Hans-Peter Jansen wrote:
$ curl -o- https://pmbs-api.links2linux.de/public
You need curl -L.
Thanks, Andreas: curl -Lo- https://pmbs-api.links2linux.de/public <about> <title>Open Build Service API</title> <description>API to the Open Build Service</description> <revision>2.10</revision> <last_deployment>2019-07-05 14:35:37 +0200</last_deployment> <commit>1b66a6c499ef52f61406c333d92731323d852fbc</commit> </about> but the underlying problem is still unresolved. Do you have an idea, where these rpc timeouts for PMBS could stem from? Cheers, Pete -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-buildservice+owner@opensuse.org
participants (5)
-
Andreas Schwab -
Hans-Peter Jansen -
Jan Engelhardt -
Neal Gompa -
Stefan Botter