August 2011 - openSUSE Build Service - openSUSE Mailing Lists

[opensuse-buildservice] extending the /source/project backend call
by Marcus Hüwe 02 Sep '11

02 Sep '11

Hi, what about extending the /source/project backend call with an additional "withsrcmd5" query parameter? Instead of the plain package list it will return also the latest srcmd5 for each package. This call can be used by osc to check which packages in a local project have to be updated. It would probably speed up the update process for large projects because osc only needs to do one http request (instead one for each package) in order to decide if a package has to be updated or not. A drawback is that this call might be a bit "expensive" because for each package the getfilelist() function has to be called (a simple getrev() for each package won't work with links or branches). Any opinions? Marcus -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe(a)opensuse.org For additional commands, e-mail: opensuse-buildservice+help(a)opensuse.org

2 6

[opensuse-buildservice] authorization issues, help please
by Robert Schweikert 01 Sep '11

01 Sep '11

Hi, Need some help figuring this out. -> osc co home:rjschwei:openshot Server returned an error: HTTP Error 401: Authorization Required -> osc --version 0.132.5 My ~/.oscrc file contains: [https://api.opensuse.org] user = rjschwei pass = XXXXXXXXXXXXXXXXXXXXX I have the same .oscrc file on another machine, as well as the same version of osc and things work there????? Thanks, Robert -- Robert Schweikert MAY THE SOURCE BE WITH YOU SUSE-IBM Software Integration Center LINUX Tech Lead rjschwei(a)suse.com rschweik(a)ca.ibm.com 781-464-8147 -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe(a)opensuse.org For additional commands, e-mail: opensuse-buildservice+help(a)opensuse.org

2 4

[opensuse-buildservice] OBS Web Interface Error
by Ann Davis 31 Aug '11

31 Aug '11

Hi all, I set up a new 2.1.11 OBS instance. I can login fine, and I can create a new project. I can check out any project using osc. But whenever I try to use the webui to access _any_ project page, I get the following error: OBS Web Interface Error: Error Details: Errorcode: unknown Message: Parsing XML failed: Must specify a string with one or more characters I don't get this error on my other OBS 2.1.4 instance. Any ideas? Note: I compared the /srv/www/obs/webui/log/production.log from both instances; difference is shown below. The output seems to indicate that I left a field blank but I didn't. good log ======= [D|# 7319] http_do #1: method: get url: http://localhost:81/search/request?match=state%2F%40name%3D%27new%27+and+ac… [D|# 7319] Request took 0.009811 seconds [E|# 7319] MemCacheError (No connection to server (localhost:11211 DEAD (Errno::ECONNREFUSED: Connection refused - connect(2)), will retry at Tue Aug 30 12:07:19 -0600 2011)): No connection to server (localhost:11211 DEAD (Errno::ECONNREFUSED: Connection refused - connect(2)), will retry at Tue Aug 30 12:07:19 -0600 2011) [D|# 7319] Cache write (will save 10.16ms): Collection_39c4936b61d21b69bc031556939da7bd [D|# 7319] Person.find( 1800 seconds, andavis) [D|# 7319] Cache read: Person_9307c152a8a319a7dffa5301677b0921 bad log ======= [D|# 4476] http_do #1: method: get url: http://localhost:81/search/request?match=state%2F%40name%3D%27new%27+and+ac… [D|# 4476] Request took 0.004158 seconds [E|# 4476] MemCacheError (No connection to server (localhost:11211 DEAD (Errno::ECONNREFUSED: Connection refused - connect(2)), will retry at Tue Aug 30 16:03:16 -0400 2011)): No connection to server (localhost:11211 DEAD (Errno::ECONNREFUSED: Connection refused - connect(2)), will retry at Tue Aug 30 16:03:16 -0400 2011) [D|# 4476] Cache write (will save 4.50ms): Collection_39c4936b61d21b69bc031556939da7bd [E|# 4476] Error parsing XML: Must specify a string with one or more characters [E|# 4476] XML content was: Thanks very much (in advance) for any help. Ann -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe(a)opensuse.org For additional commands, e-mail: opensuse-buildservice+help(a)opensuse.org

2 2

[opensuse-buildservice] deploying the OBS backend to multiple servers
by Robert Schiele 31 Aug '11

31 Aug '11

Hi all, I was doing some thoughts about what is the best way for load balancing by deploying the backend to multiple servers. I have given this some thoughts and would be interested to hear the opinion of people with experience on deploying things to multiple servers or by those that actually implemented those things. My understanding is that in principle I could deploy each and every backend module (like bs_dispatcher or bs_publish) could be deployed to a different system as long as they have a shared /srv/obs directory. I am not saying that doing things in that an extreme way is reasonable but just wanted to check whether this is technically correct or whether someone sees a problem with that? Next I did a quick analysis of what backend module seems to access which parts of the /srv/obs namespace and found the following relations: bs_srcserver reads and writes to /build, /db/published, /db/source, /db/request, /diffcache, /projects, /remotecache, /sources, /trees, and /events/service bs_repserver reads and writes to /build, /workers, /repos, /events/$ARCH, /events/dispatch, and /events/signer bs_repserver reads from /db/published, /info, and /events/publish bs_scheduler reads and writes to /jobs, /info, and /events/$ARCH bs_scheduler reads from /repos bs_dispatcher reads and writes to /workers, /jobs, /events/dispatch, and /events/repository bs_warden reads and writes to /workers and /jobs bs_signer reads and writes to /jobs, /events/$ARCH, and /events/signer bs_publisher reads and writes to /build, /db/published, /repos, /repos_sync, and /events/publish bs_sshgit reads and writes to /projects This information was gathered by a lot of grepping through the source code and thus is not necessarily complete or completely correct. What I'd be interested here is now whether someone could tell me whether this is correct or what is missing/wrong in this analysis. What is completely missing from this analysis is the /upload directory, since I saw that multiple modules were using that one but it seemed they were using completely independent namespaces within there. Is this correct? The background of this analysis is that if I splitted different modules to different servers that I knew which directories should be shared among which hosts via NFS and which of them could be on the local disk. Or would you generally recommend to share the whole filesystem space among them since relying on the above information is generally to fragile? Are there people with a general recommendation on how to split the various modules to different servers. When I talked to Adrian once he suggested to run the WebUI+API on one machine, the srcserver on another one and the rest on a third one (at least if I remember correctly). For sure the workers will run on separate machines in any case. Is this what people think is a good balancing or does someone come up with different ideas? Could someone describe the infrastructure of build.opensuse.org? Which parts are running on separate servers, what parts of the relevant filesystems are network mounted on the machines from where and so on? Robert BTW: I created a graph of the information mentioned above with graphviz. If someone with better insight can provide the missing pieces and fixes and people think this is interesting for OBS documentation I am happy to contribute that. -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe(a)opensuse.org For additional commands, e-mail: opensuse-buildservice+help(a)opensuse.org

2 3

[opensuse-buildservice] How to clean up /srv/obs/sources?
by Carsten Hoeger 31 Aug '11

31 Aug '11

Hi, I just noticed that I have ~6GB in repositories (binary and source rpms/deps), but my obs uses ~600GB on disk... Then I found the culprit. It's the /srv/obs/sources folder: 6.4G repos 521G sources Okay, now how to get rid of the useless old stuff? My disk is again filling up... :-( -- With best regards, Carsten Hoeger

5 7

[opensuse-buildservice] [PATCH 0/3] Fixes to obs-service-tar_scm
by Markus Lehtonen 29 Aug '11

29 Aug '11

Three minor fixes to obs-service-tar_scm to work better when an existing source tarball is updated. The third patch requires obs-server v2.3 (more specifically commit 9647a4ed in upstream) to work correctly. Markus Lehtonen (3): [obs-service-tar_scm] Checkout master before git update [obs-service-tar_scm] create tarball even if version does not change [obs-service-tar_scm] Grab (possibly) existing tar from .old subdir tar_scm | 24 +++--------------------- 1 files changed, 3 insertions(+), 21 deletions(-) -- 1.7.3.4 -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe(a)opensuse.org For additional commands, e-mail: opensuse-buildservice+help(a)opensuse.org

2 11

[opensuse-buildservice] Build Service & upstream documentation patches
by Michal Hrusecky 29 Aug '11

29 Aug '11

Hi, apart from the conference, last of the things we were focusing to in openSUSE Boosters team was to make it for upstream projects easier to use openSUSE Build Service for publishing binaries of their software. Part of it was documenting how upstream can use OBS. So here comes few patches for OBS Best Practices book. Rendered version is available temporally on my server[1] (mainly upstream part[2] that changed) for more convenient reading. Feedback and potential integration will be appreciated. [1] https://michal.hrusecky.net/tmp/obs-best-practices/ [2] https://michal.hrusecky.net/tmp/obs-best-practices/cha.obs.best-practices.u… -- Michal HRUSECKY SUSE LINUX, s.r.o openSUSE Boosters Team Lihovarska 1060/12 PGP 0xFED656F6 19000 Praha 9 mhrusecky[at]suse.cz Czech Republic http://michal.hrusecky.net http://www.suse.cz

1 0

[opensuse-buildservice] obs-service-download_files and Sourceforge
by Cristian Morales Vega 29 Aug '11

29 Aug '11

I used this kind of source tags with SourceForge: Source0: http://downloads.sourceforge.net/project/nsnam/ns-2/%{version}/ns-%{version… And the # Some web sites need a special user agent if echo $url | egrep -q '^http://[^/]*sourceforge.net/'; then # default wget user agent required, but /download suffix must be added urlextension="/download" else part of the download_files service is breaking it for me. What's the logic behind appending "/download"? -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe(a)opensuse.org For additional commands, e-mail: opensuse-buildservice+help(a)opensuse.org

2 3

[opensuse-buildservice] saving OBS resources by not building unchanged platforms ?
by Martin Koller 29 Aug '11

29 Aug '11

Hi, my package showimage https://build.opensuse.org/package/show?package=showimage&project=home%3Ama… can build rpm and also deb packages now. While modifying the debian.rules file (via the web interface) I discovered that also all rpm packages are rebuilt. Wouldn't it be better to avoid this and only rebuild deb packages when any of the debian/dsc files are changed and only rpm packages are built when changing the spec file (and of course everything when changing the source package) ? -- Best regards/Schöne Grüße Martin A: Because it breaks the logical sequence of discussion Q: Why is top posting bad? () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments Geschenkideen, Accessoires, Seifen, Kulinarisches: www.bibibest.at

1 0

[opensuse-buildservice] Unable to connect to API host error
by Ann Davis 29 Aug '11

29 Aug '11

Hello all, I'm setting up a new OBS 2.1.11 instance using apache2 w/ vhosts as specified in http://gitorious.org/opensuse/build-service/blobs/raw/master/dist/README.SE… . When I access https://localhost , I get the message: OBS Web Interface Error: Error Details: Errorcode: 503 Message: Unable to connect to API host (localhost) but if I access https://localhost:444 , the OBS API page comes up fine (after a login prompt). I added the line 127.0.0.1 api to my /etc/hosts file but this doesn't seem to make any difference. (Note: the http://gitorious.org/opensuse/build-service/blobs/raw/master/dist/README.SE… differs from the /usr/share/doc/packages/obs-api/README.SETUP installed by the ops-api package. The one from the package instructs to use lighttpd instead of apache2. My other OBS 2.1.4 instance is using lighttpd and it's all working fine.) Any ideas? Thanks in advance, Ann -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe(a)opensuse.org For additional commands, e-mail: opensuse-buildservice+help(a)opensuse.org

2 1