[yast-devel] Migrate YaST svn to git
Dear YaST-Hackers, As we heared from various sides about a switch of the YaST source code from SVN to git - here is all information about it. History -------- Last year we (Klaus Kämpf, Lukas Ocilka, Martin Vidner and me) started to look into a migration of the YaST svn repo to git. We did not succeed back then, as our usage of svn is not that simple as in other repos. I already migrated smaller projects with git-svn within a few minutes - but this approach did not work for YaST. The topic popped up again recently and Bernhard Wiedemann and me looked into it again last week and found a new solution. Within 1.5 days we created a tool around svndumpfilter3 [1] that looks very promising. Our Goals ---------- In long discussions in 2011 the initiators agreed on these set of goals that we should meet: * have one git repo per YaST module * have all history in the repos including all branches and tags * have a kind of yast-meta repo containing scripts that are able to checkout single or all yast repos in the same style like the current svn checkout or to search within the other repos Thomas Göttlicher <tgoettlicher@suse.de> is collecting your usecases (like 'find latest version tag of package X in distribution Y', ...) * cleanup and unify the branch and tag names Find all details about it at: http://en.opensuse.org/YaST_SVN_to_GIT * automatically move all repos to github.com/yast/* Current Status --------------- We have some of the modules converted and published for you as a PREview. http://www3.zq1.de/bernhard/linux/yast/ Please have a look at your module and report issues to this list or directly to Bernhard and me (bwiedemann@suse.de , jdsn@suse.de). Please do NOT push these repos to github or use them for development now. These are only published for you to get a first impression and to find errors. Once everything works, we will create repos on and push the final repos to github, all automatically. Thus don't even create a repo there yet. Due to proirities Bernhard and me work on this task in our free time. But we already got so far that we think that we are able to finish the conversion within a few days (see plan below). Anyhow we do not have a fixed date yet. We have to solve a hand full of issues to make the conversion complete. Final Migration ----------------- This is our plan for the final migration once our tests succeed we will 1. announce the final switch at least two days beforehand 2. switch the svn repo to read-only on 'a' Friday afternoon 3. create a final svn dump 4. run the conversion tool (this is the most time consuming job) 5. automatically create repos on github/yast/* 6. upload the final git repos If all works well, you can find an announcement of the conversion in your inbox on Monday morning. We do not have a date yet for the conversion. So there will be another announcement. If there are any blockers or issues on your side please speak up or contact Bernhard and me (see above). Any other kind of feedback is welcome as well. [1] http://furius.ca/pubcode/pub/conf/bin/svndumpfilter3.html Ciao, Daniel -- J. Daniel Schmidt <jdsn@suse.de> SUSE LINUX Products GmbH Research & Development Maxfeldstr. 5 HRB 16746 (AG Nürnberg) D-90409 Nürnberg GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Thu, 23 Feb 2012 02:05:11 +0100 "J. Daniel Schmidt" <jdsn@suse.de> wrote:
Find all details about it at: http://en.opensuse.org/YaST_SVN_to_GIT
Moved to http://en.opensuse.org/openSUSE:YaST_SVN_to_GIT , but redirect will work for original link. Main namespace is used for distro presentation articles, and YaST_SVN_to_GIT is work planing artcile which by definition fits perfectly to openSUSE: namespace. openSUSE:YaST_SVN_to_GIT was added to category YaST. -- Regards, Rajko -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
* J. Daniel Schmidt <jdsn@suse.de> [Feb 23. 2012 02:05]:
History -------- Last year we (Klaus Kämpf, Lukas Ocilka, Martin Vidner and me) started to look into a migration of the YaST svn repo to git. We did not succeed back then, as our usage of svn is not that simple as in other repos. I already migrated smaller projects with git-svn within a few minutes - but this approach did not work for YaST.
The topic popped up again recently and Bernhard Wiedemann and me looked into it again last week and found a new solution. Within 1.5 days we created a tool around svndumpfilter3 [1] that looks very promising.
Thats great progress ! Just keep in mind that all previous attempts looked simple for most of the existing YaST modules. However, some modules have an - 'interesting' - branch/merge history and the devil is in the details. Klaus --- SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Maxfeldstraße 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 23.2.2012 07:46, Klaus Kaempf wrote:
* J. Daniel Schmidt<jdsn@suse.de> [Feb 23. 2012 02:05]:
History -------- Last year we (Klaus Kämpf, Lukas Ocilka, Martin Vidner and me) started to look into a migration of the YaST svn repo to git. We did not succeed back then, as our usage of svn is not that simple as in other repos. I already migrated smaller projects with git-svn within a few minutes - but this approach did not work for YaST.
The topic popped up again recently and Bernhard Wiedemann and me looked into it again last week and found a new solution. Within 1.5 days we created a tool around svndumpfilter3 [1] that looks very promising.
Thats great progress !
Just keep in mind that all previous attempts looked simple for most of the existing YaST modules. However, some modules have an - 'interesting' - branch/merge history and the devil is in the details.
That's great news! Thanks guys for looking into that. Lukas -- Lukas Ocilka, Appliances Department, SUSE LINUX s.r.o. MD: Jeff Hawn, Jennifer Guild, Alena Hendrichova -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Hi, On Thu, Feb 23, J. Daniel Schmidt wrote:
Current Status --------------- We have some of the modules converted and published for you as a PREview. http://www3.zq1.de/bernhard/linux/yast/
Please have a look at your module and report issues to this list or directly to Bernhard and me (bwiedemann@suse.de , jdsn@suse.de).
Arvin and me had a short look at yast2-storage. So far all looks fine - good job. One thing is missing. The property svn:ignore is not migrated. AFAIK it would be enough to do a "svn propget svn:ignore" and put the content (if nonempty) into a file .gitignore in the same dir in git repo. I think it is more useful to do that once in the migration script instead of everyone has to do it for his git repo afterwards. Tschuess, Thomas Fehr -- Thomas Fehr, SuSE Linux Products GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Tel: +49-911-74053-0, Fax: +49-911-74053-482, Email: fehr@suse.de GPG public key available. -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Thursday 23 February 2012 11:36:54 Thomas Fehr wrote:
One thing is missing. The property svn:ignore is not migrated.
AFAIK it would be enough to do a "svn propget svn:ignore" and put the content (if nonempty) into a file .gitignore in the same dir in git repo. I think it is more useful to do that once in the migration script instead of everyone has to do it for his git repo afterwards.
Yes indeed, we can and should automate this as well. Thanks for the hint, we will include it in our tool. Ciao, Daniel -- J. Daniel Schmidt <jdsn@suse.de> SUSE LINUX Products GmbH Research & Development Maxfeldstr. 5 HRB 16746 (AG Nürnberg) D-90409 Nürnberg GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 02/24/2012 11:03 AM, J. Daniel Schmidt wrote:
On Thursday 23 February 2012 11:36:54 Thomas Fehr wrote:
One thing is missing. The property svn:ignore is not migrated.
AFAIK it would be enough to do a "svn propget svn:ignore" and put the content (if nonempty) into a file .gitignore in the same dir in git repo. I think it is more useful to do that once in the migration script instead of everyone has to do it for his git repo afterwards.
Yes indeed, we can and should automate this as well. Thanks for the hint, we will include it in our tool.
Hi, I've just "finished" a simple script that checks the current YaST Git snapshot for: - number of branches - number of tags - number of commits in the current branch The script in question can be found here: https://github.com/kobliha/yast_svn_2_git_check The latest script output here: https://github.com/kobliha/yast_svn_2_git_check/blob/master/last.log Bye Lukas -- Lukas Ocilka, Appliances Department SUSE LINUX s.r.o., Praha -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 02/24/2012 03:26 PM, Lukas Ocilka wrote:
I've just "finished" a simple script that checks the current YaST Git snapshot for:
- number of branches - number of tags - number of commits in the current branch
The script in question can be found here: https://github.com/kobliha/yast_svn_2_git_check
The latest script output here: https://github.com/kobliha/yast_svn_2_git_check/blob/master/last.log
I've enhanced that tool in a way that it currently reports - Discontinued modules (INFO) - No branches available (ERROR) - Only one branch available (WARNING) - No tags available (WARNING) List of discontinued modules is in discontinued_modules file To check for issues, run `grep "ERROR\|WARNING" last.log` The current status is: (yast-OPENSSL) ERROR: No branches available (yast-SLE-11-WebYaST) ERROR: No branches available (yast-SLE-11-WebYaST) WARNING: No tags available (yast-branding) WARNING: No tags available (yast-db2) ERROR: No branches available (yast-dbus-client) WARNING: No tags available (yast-dbus-server) WARNING: No tags available (yast-doc) WARNING: No tags available (yast-drbd) WARNING: No tags available (yast-extra-packages) WARNING: No tags available (yast-faxserver) ERROR: No branches available (yast-firewallcd-admin) ERROR: No branches available (yast-hpc) WARNING: Only one branch available (yast-installation-sources) ERROR: No branches available (yast-installer) ERROR: No branches available (yast-installer) WARNING: No tags available (yast-libyui-bindings) WARNING: No tags available (yast-libyui) ERROR: No branches available (yast-libyui) WARNING: No tags available (yast-modules) WARNING: No tags available (yast-nfscim) ERROR: No branches available (yast-nfscim) WARNING: No tags available (yast-openvas-security-scanner) WARNING: Only one branch available (yast-openwsman-yast) WARNING: No tags available (yast-printerdb) ERROR: No branches available (yast-qt-graph) WARNING: No tags available (yast-qt-pkg) ERROR: No branches available (yast-qt-pkg) WARNING: No tags available (yast-qt3) ERROR: No branches available (yast-qt3) WARNING: No tags available (yast-rules) ERROR: No branches available (yast-rules) WARNING: No tags available (yast-theme-ul) ERROR: No branches available (yast-ui_qt_tests) WARNING: No tags available (yast-watcher) ERROR: No branches available (yast-web-rails) ERROR: No branches available (yast-web-rails) WARNING: No tags available (yast-web-service) ERROR: No branches available (yast-web-service) WARNING: No tags available (yast-web-wt) ERROR: No branches available (yast-web-wt) WARNING: No tags available (yast-wlanipsecclient) ERROR: No branches available (yast-x11.sav) ERROR: No branches available (yast-x11.sav) WARNING: No tags available - looks like a mistake (yast-y2khtml) ERROR: No branches available (yast-yxmlconv) WARNING: No tags available Bye Lukas -- Lukas Ocilka, Appliances Department SUSE LINUX s.r.o., Praha -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Hello, On Feb 28 11:08 Lukas Ocilka wrote (excerpt):
To check for issues, run `grep "ERROR\|WARNING" last.log` ... (yast-printerdb) ERROR: No branches available
yast-printerdb or yast2-printerdb is from stone-age and not in any of our maintained products, in particular it is not in SLE9. But I think it was used in SLE8. Regarding migrating YaST svn to git please ignore yast-printerdb. Kind Regards Johannes Meixner -- SUSE LINUX Products GmbH -- Maxfeldstrasse 5 -- 90409 Nuernberg -- Germany HRB 16746 (AG Nuernberg) GF: Jeff Hawn, Jennifer Guild, Felix Imendoerffer -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 02/23/2012 02:05 AM, J. Daniel Schmidt wrote:
* have all history in the repos including all branches and tags
* have a kind of yast-meta repo containing scripts that are able to checkout single or all yast repos in the same style like the current svn checkout or to search within the other repos Thomas Göttlicher<tgoettlicher@suse.de> is collecting your usecases (like 'find latest version tag of package X in distribution Y', ...)
There are two big projects doing something in this area which are _not_ using git submodules because they are ... weird. Android: repo tool: http://source.android.com/source/version-control.html http://source.android.com/source/using-repo.html Android has hundred of gits repos of various components and they use the repo tool to work with them. They can tag this way a release across all repositories for example. Chromium uses gclient from the depot tools http://dev.chromium.org/developers/how-tos/depottools (only feature, it works on Windows) -- Duncan Mac-Vicar P. - http://www.suse.com/ SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Maxfeldstraße 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Monday 27 February 2012 15:38:25 Duncan Mac-Vicar P. wrote:
* have a kind of yast-meta repo containing scripts that are able to checkout> single or all yast repos in the same style like the current svn checkout or to search within the other repos Thomas Göttlicher<tgoettlicher@suse.de> is collecting your usecases (like 'find latest version tag of package X in distribution Y', ...)
There are two big projects doing something in this area which are _not_ using git submodules because they are ... weird.
We do not intend to use git submodules as we know about the difficulties. There even is no need to use them at all. The script in the yast-meta package (name not yet choosen) just knows about all the other yast repos (either statically with a list in a file, or dynamically by querying the github api). The only magic the script does, is to checkout the repos in the same directory structure as we have currently with the svn repo. Its just to make the switch a little easier. So nobody has to use the meta package, but it might help people who are used to our current svn layout. The script currently only exists in my head. I will write a first version within the next days and hope it will be improved and extended by many yasties. Ciao, Daniel -- J. Daniel Schmidt <jdsn@suse.de> SUSE LINUX Products GmbH Research & Development Maxfeldstr. 5 HRB 16746 (AG Nürnberg) D-90409 Nürnberg GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 02/27/2012 08:19 PM, J. Daniel Schmidt wrote:
So nobody has to use the meta package, but it might help people who are used to our current svn layout.
The script currently only exists in my head. I will write a first version within the next days and hope it will be improved and extended by many yasties.
Ciao, Daniel
The repo tool serves a different purpose. You can work in various modules at the same time and see what is changed and/or commit in one shot to different git repositories. At the same time, the manifest in the meta-repository, like submodules, points to specific heads in the individual repositories), therefore you can checkout "openSUSE_11_3" tag/branch in one shot as well. -- Duncan Mac-Vicar P. - http://www.suse.com/ SUSE LINUX Products GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 16746 (AG Nürnberg) Maxfeldstraße 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
participants (7)
-
Duncan Mac-Vicar P. -
J. Daniel Schmidt -
Johannes Meixner -
Klaus Kaempf -
Lukas Ocilka -
Rajko M. -
Thomas Fehr