[opensuse-buildservice] Re: [PATCH] git_pull service osc support
Please adapt this to use "tar_scm" service instead. I do not want to have two services by default for the same thing. (Please see my other mail). bye adrian Am Montag, 20. September 2010, 11:06:47 schrieb OBS osc:
From: brook hong
--- osc/commandline.py | 5 ++++- osc/core.py | 27 +++++++++++++++++++++++++++ 2 files changed, 31 insertions(+), 1 deletions(-)
diff --git a/osc/commandline.py b/osc/commandline.py index 0e11913..d5432b3 100644 --- a/osc/commandline.py +++ b/osc/commandline.py @@ -2880,7 +2880,10 @@ Please submit there instead, or use --nodevelproject to force direct submission. # Do some magic here, when adding a url. We want that the server to download the tar ball and to verify it for arg in parseargs(args): if arg.startswith('http://') or arg.startswith('https://') or arg.startswith('ftp://'): - addDownloadUrlService(arg) + if arg.endswith('.git'): + addGitSource(arg) + else: + addDownloadUrlService(arg) else: addFiles([arg])
diff --git a/osc/core.py b/osc/core.py index 1bead66..47124ea 100644 --- a/osc/core.py +++ b/osc/core.py @@ -301,6 +301,12 @@ class Serviceinfo: r.append( s ) return r
+ def addGitUrl(self, serviceinfo_node, url_string): + r = serviceinfo_node + s = ET.Element( "service", name="git_pull" ) + ET.SubElement(s, "param", name="url").text = url_string + r.append( s ) + return r
def execute(self, dir): import tempfile @@ -5169,6 +5175,27 @@ def stripETxml(node): for child in node.getchildren(): stripETxml(child)
+def addGitSource(url): + service_file = os.path.join(os.getcwd(), '_service') + addfile = False + if os.path.exists( service_file ): + services = ET.parse(os.path.join(os.getcwd(), '_service')).getroot() + else: + services = ET.fromstring("<services />") + addfile = True + stripETxml( services ) + si = Serviceinfo() + s = si.addGitUrl(services, url) + si.read(s) + + # for pretty output + reparsed = minidom.parseString(ET.tostring(s)) + f = open(service_file, 'wb') + f.write(reparsed.toprettyxml(indent=" ")) + f.close() + if addfile: + addFiles( ['_service'] ) + def addDownloadUrlService(url): service_file = os.path.join(os.getcwd(), '_service') addfile = False
-- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Hi, Ok, I'll adapt osc to use "tar_scm" service. But how to use extrace_file service? For example, after using "tar_scm" service, I can get a _service:tar_scm:libzip-1282898421.tar from http://git.gitorious.org/libzip/libzip.git, And the filename of the tar file changes each time. How to pass the exact file name to extract_file service? Since I need extract a spec file from the tar file. ---------------- Best Regards, Brook Hong
-----Original Message----- From: ext Adrian Schröter [mailto:adrian@suse.de] Sent: Monday, September 20, 2010 5:46 PM To: OBS osc Cc: dl9pf@gmx.de; mls@suse.de; ludwig.nussel@suse.de; tschmidt@suse.de; coolo@kde.org; mvyskocil@suse.cz; vuntz@gnome.org; bg@suse.de; freitag@suse.de; jfunk@funktronics.ca; suse-tux@gmx.de; ro@suse.de; david@dgreaves.com; thomas@suse.de; nashif@linux.intel.com Subject: [opensuse-buildservice] Re: [PATCH] git_pull service osc support
Please adapt this to use "tar_scm" service instead.
I do not want to have two services by default for the same thing. (Please see my other mail).
bye adrian
Am Montag, 20. September 2010, 11:06:47 schrieb OBS osc:
From: brook hong
--- osc/commandline.py | 5 ++++- osc/core.py | 27 +++++++++++++++++++++++++++ 2 files changed, 31 insertions(+), 1 deletions(-)
diff --git a/osc/commandline.py b/osc/commandline.py index 0e11913..d5432b3 100644 --- a/osc/commandline.py +++ b/osc/commandline.py @@ -2880,7 +2880,10 @@ Please submit there instead, or use -- nodevelproject to force direct submission. # Do some magic here, when adding a url. We want that the server to download the tar ball and to verify it for arg in parseargs(args): if arg.startswith('http://') or arg.startswith('https://') or arg.startswith('ftp://'): - addDownloadUrlService(arg) + if arg.endswith('.git'): + addGitSource(arg) + else: + addDownloadUrlService(arg) else: addFiles([arg])
diff --git a/osc/core.py b/osc/core.py index 1bead66..47124ea 100644 --- a/osc/core.py +++ b/osc/core.py @@ -301,6 +301,12 @@ class Serviceinfo: r.append( s ) return r
+ def addGitUrl(self, serviceinfo_node, url_string): + r = serviceinfo_node + s = ET.Element( "service", name="git_pull" ) + ET.SubElement(s, "param", name="url").text = url_string + r.append( s ) + return r
def execute(self, dir): import tempfile @@ -5169,6 +5175,27 @@ def stripETxml(node): for child in node.getchildren(): stripETxml(child)
+def addGitSource(url): + service_file = os.path.join(os.getcwd(), '_service') + addfile = False + if os.path.exists( service_file ): + services = ET.parse(os.path.join(os.getcwd(), '_service')).getroot() + else: + services = ET.fromstring("<services />") + addfile = True + stripETxml( services ) + si = Serviceinfo() + s = si.addGitUrl(services, url) + si.read(s) + + # for pretty output + reparsed = minidom.parseString(ET.tostring(s)) + f = open(service_file, 'wb') + f.write(reparsed.toprettyxml(indent=" ")) + f.close() + if addfile: + addFiles( ['_service'] ) + def addDownloadUrlService(url): service_file = os.path.join(os.getcwd(), '_service') addfile = False
-- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Am Samstag, 25. September 2010, 05:57:26 schrieb brook.hong@nokia.com:
Hi,
Ok, I'll adapt osc to use "tar_scm" service. But how to use extrace_file service?
For example, after using "tar_scm" service, I can get a _service:tar_scm:libzip-1282898421.tar from http://git.gitorious.org/libzip/libzip.git, And the filename of the tar file changes each time.
How to pass the exact file name to extract_file service? Since I need extract a spec file from the tar file.
You can use regexp, so for example "libzip*tar*". But maybe it would make sense to extend tar_scm to copy out defined files directly to avoid the extra extraction step. -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Hi, There seems to be a little bug in the extract_file script. Bs_service launches the shell script from $myworkdir, and the tar file generated by tar_scm was put under $myworkdir before hand. But there is a `cd "$MYOUTDIR"` in extract_file, $MYOUTDIR was set "$myworkdir/.tmp", and this causes extract_file not finding a tar file. A feasible solution is to put "MYARCHIVE=`pwd`/$MYARCHIVE" before the cd operation. -----Original Message----- From: Hong Brook (Nokia-MS/Beijing) Sent: Saturday, September 25, 2010 11:57 AM To: adrian@suse.de; opensuse-buildservice@opensuse.org Cc: dl9pf@gmx.de; mls@suse.de; ludwig.nussel@suse.de; tschmidt@suse.de; coolo@kde.org; mvyskocil@suse.cz; vuntz@gnome.org; bg@suse.de; freitag@suse.de; jfunk@funktronics.ca; suse-tux@gmx.de; ro@suse.de; david@dgreaves.com; thomas@suse.de; nashif@linux.intel.com Subject: RE: [opensuse-buildservice] Re: [PATCH] git_pull service osc support Hi, Ok, I'll adapt osc to use "tar_scm" service. But how to use extrace_file service? For example, after using "tar_scm" service, I can get a _service:tar_scm:libzip-1282898421.tar from http://git.gitorious.org/libzip/libzip.git, And the filename of the tar file changes each time. How to pass the exact file name to extract_file service? Since I need extract a spec file from the tar file. ---------------- Best Regards, Brook Hong
-----Original Message----- From: ext Adrian Schröter [mailto:adrian@suse.de] Sent: Monday, September 20, 2010 5:46 PM To: OBS osc Cc: dl9pf@gmx.de; mls@suse.de; ludwig.nussel@suse.de; tschmidt@suse.de; coolo@kde.org; mvyskocil@suse.cz; vuntz@gnome.org; bg@suse.de; freitag@suse.de; jfunk@funktronics.ca; suse-tux@gmx.de; ro@suse.de; david@dgreaves.com; thomas@suse.de; nashif@linux.intel.com Subject: [opensuse-buildservice] Re: [PATCH] git_pull service osc support
Please adapt this to use "tar_scm" service instead.
I do not want to have two services by default for the same thing. (Please see my other mail).
bye adrian
Am Montag, 20. September 2010, 11:06:47 schrieb OBS osc:
From: brook hong
--- osc/commandline.py | 5 ++++- osc/core.py | 27 +++++++++++++++++++++++++++ 2 files changed, 31 insertions(+), 1 deletions(-)
diff --git a/osc/commandline.py b/osc/commandline.py index 0e11913..d5432b3 100644 --- a/osc/commandline.py +++ b/osc/commandline.py @@ -2880,7 +2880,10 @@ Please submit there instead, or use -- nodevelproject to force direct submission. # Do some magic here, when adding a url. We want that the server to download the tar ball and to verify it for arg in parseargs(args): if arg.startswith('http://') or arg.startswith('https://') or arg.startswith('ftp://'): - addDownloadUrlService(arg) + if arg.endswith('.git'): + addGitSource(arg) + else: + addDownloadUrlService(arg) else: addFiles([arg])
diff --git a/osc/core.py b/osc/core.py index 1bead66..47124ea 100644 --- a/osc/core.py +++ b/osc/core.py @@ -301,6 +301,12 @@ class Serviceinfo: r.append( s ) return r
+ def addGitUrl(self, serviceinfo_node, url_string): + r = serviceinfo_node + s = ET.Element( "service", name="git_pull" ) + ET.SubElement(s, "param", name="url").text = url_string + r.append( s ) + return r
def execute(self, dir): import tempfile @@ -5169,6 +5175,27 @@ def stripETxml(node): for child in node.getchildren(): stripETxml(child)
+def addGitSource(url): + service_file = os.path.join(os.getcwd(), '_service') + addfile = False + if os.path.exists( service_file ): + services = ET.parse(os.path.join(os.getcwd(), '_service')).getroot() + else: + services = ET.fromstring("<services />") + addfile = True + stripETxml( services ) + si = Serviceinfo() + s = si.addGitUrl(services, url) + si.read(s) + + # for pretty output + reparsed = minidom.parseString(ET.tostring(s)) + f = open(service_file, 'wb') + f.write(reparsed.toprettyxml(indent=" ")) + f.close() + if addfile: + addFiles( ['_service'] ) + def addDownloadUrlService(url): service_file = os.path.join(os.getcwd(), '_service') addfile = False
-- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
You can use regexp, so for example "libzip*tar*".
But maybe it would make sense to extend tar_scm to copy out defined files directly to avoid the extra extraction step. That's good.
And I find another issue in the extract_file script just now. With tar_scm, a directory is added into the tar file as a root. In the extract_file script, it uses `tar xfj "$existing_archive" $MYFILES`. Then a path must be included in $MYFILES. But the path was removed by a ## pattern in the script. Thus extract_file can never extract file from the tar file generated by tar_scm. Adrian, will you or anyone else fix this issue and another issue(mentioned in another mail)? Or extend tar_scm?
-- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Am Dienstag, 28. September 2010, 08:23:56 schrieb brook.hong@nokia.com:
You can use regexp, so for example "libzip*tar*".
But maybe it would make sense to extend tar_scm to copy out defined files directly to avoid the extra extraction step. That's good.
And I find another issue in the extract_file script just now. With tar_scm, a directory is added into the tar file as a root. In the extract_file script, it uses `tar xfj "$existing_archive" $MYFILES`. Then a path must be included in $MYFILES. But the path was removed by a ## pattern in the script.
Yes, there is also a general problem with bs_service daemon which leads to the problem that service X can not obsolete files from service Y in all cases. That is something I need to fix first for 2.1. To support that, I plan also to modify the naming of the files from _service:$SERVICE:$FILE to _service:$FILE
Thus extract_file can never extract file from the tar file generated by tar_scm.
Adrian, will you or anyone else fix this issue and another issue(mentioned in another mail)? Or extend tar_scm?
I would be happy to extend tar_scm, but I am time limited atm. So it would be great if you can provide a patch (or direct submitrequest to openSUSE:Tools:Unstable). There is also the request to clone files directly without tar-ing them (to support distros who maintain their files in svn/git directly). Therefore I was also considering to rename tar_scm to something like "download_scm" and add the tar functionlity optional to it. Because I like to avoid to copy the git/svn functionality into another service. Would be great if you or anyone else could think about this approach and deliver a solution. thanks adrian -- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de -- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
Adrian, will you or anyone else fix this issue and another issue(mentioned in another mail)? Or extend tar_scm?
I would be happy to extend tar_scm, but I am time limited atm. So it would be great if you can provide a patch (or direct submitrequest to openSUSE:Tools:Unstable).
There is also the request to clone files directly without tar-ing them (to support distros who maintain their files in svn/git directly).
Therefore I was also considering to rename tar_scm to something like "download_scm" and add the tar functionlity optional to it. Because I like to avoid to copy the git/svn functionality into another service.
Would be great if you or anyone else could think about this approach and deliver a solution.
Great, I'll do this. But we'll have 7-days holiday soon, I'm not sure if I can finish it before my holiday.
thanks adrian
-- Adrian Schroeter SUSE Linux Products GmbH email: adrian@suse.de
-- To unsubscribe, e-mail: opensuse-buildservice+unsubscribe@opensuse.org For additional commands, e-mail: opensuse-buildservice+help@opensuse.org
participants (2)
-
Adrian Schröter -
brook.hong＠nokia.com