[yast-devel] how to mark *.ybc file outdated?
Hello RPMs for YaST modules install *.ycp files and *.ybc files. Is it possible to change a *.ycp file and then tell YaST somehow that the matching *.ybc file is outdated so that the new code in the new *.ycp file is used? In other words: What must I do to force YaST to ignore the old *.ybc file and use the new *.ycp file or how can I create a single new *.ybc file from a single new *.ycp file with a usual openSUSE 10.3 default installation (i.e. no development tools are installed on this system)? Background information: Regarding bug 310177 I liked to try out a small patch. Therefore I canged /usr/share/YaST2/modules/Printerdb.ycp But I cannot test it because when I change /usr/share/YaST2/modules/Printerdb.ycp it has no effect on the running YaST printer module. It seems YaST uses always /usr/share/YaST2/modules/Printerdb.ybc regardless if the associated *.ycp file has changed. I tried to simply move /usr/share/YaST2/modules/Printerdb.ybc away, but then the printer module cannot be started at all. I get "No such client module printer". Kind Regards Johannes Meixner -- SUSE LINUX Products GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany AG Nuernberg, HRB 16746, GF: Markus Rex -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
Johannes Meixner wrote:
Hello
RPMs for YaST modules install *.ycp files and *.ybc files.
Is it possible to change a *.ycp file and then tell YaST somehow that the matching *.ybc file is outdated so that the new code in the new *.ycp file is used?
In other words: What must I do to force YaST to ignore the old *.ybc file and use the new *.ycp file or how can I create a single new *.ybc file from a single new *.ycp file with a usual openSUSE 10.3 default installation (i.e. no development tools are installed on this system)?
YaST always uses only the YBC format of the file, YCP is there only as a reference. You can easily make a YBC from YCP file with this command: ycpc -c $file.ycp Call 'ycpc --help' for more options. Lukas
Dňa Friday 14 September 2007 09:59:23 Lukas Ocilka ste napísal:
Johannes Meixner wrote:
Hello
RPMs for YaST modules install *.ycp files and *.ybc files.
Is it possible to change a *.ycp file and then tell YaST somehow that the matching *.ybc file is outdated so that the new code in the new *.ycp file is used?
In other words: What must I do to force YaST to ignore the old *.ybc file and use the new *.ycp file or how can I create a single new *.ybc file from a single new *.ycp file with a usual openSUSE 10.3 default installation (i.e. no development tools are installed on this system)?
YaST always uses only the YBC format of the file, YCP is there only as a reference.
You can easily make a YBC from YCP file with this command: ycpc -c $file.ycp
Call 'ycpc --help' for more options.
This will not help if you change the public interface of your module. Stano -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
On 14/09/2007, Johannes Meixner <jsmeix@suse.de> wrote:
I tried to simply move /usr/share/YaST2/modules/Printerdb.ybc away, but then the printer module cannot be started at all. I get "No such client module printer".
Did you try recreating the ybc with "ycpc -c /usr/share/YaST2/modules/Printerdb.ycp" ? _ Benjamin Weber -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
On Friday 14 September 2007 09:52, Johannes Meixner wrote: [...byte code files outdated...] I've been wondering for a long time if we shouldn't seriously consider to handle the byte code the same way Perl does: Byte-compile "on the fly" and keep the byte code just in memory, never write it to disk. IMHO this would make handling YCP code _much_ easier. I have no clue just how much the overall performance gain due to byte code is at all. It saves a syntax check for the byte-compiled code. Loops might be faster. But we don't byte-compile everything anyway, just modules (those that can be imported). So exactly how big is the performance difference? Is it worth all the hassle we have with the .ybc files? (Just think about build order etc.) Maybe a topic for the upcoming YaST2 workshop. CU -- Stefan Hundhammer <sh@suse.de> Penguin by conviction. YaST2 Development SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) Nürnberg, Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
* Stefan Hundhammer <sh@suse.de> [Sep 14. 2007 11:51]:
On Friday 14 September 2007 09:52, Johannes Meixner wrote: [...byte code files outdated...]
I've been wondering for a long time if we shouldn't seriously consider to handle the byte code the same way Perl does: Byte-compile "on the fly" and keep the byte code just in memory, never write it to disk. IMHO this would make handling YCP code _much_ easier.
What are the problems with the current way of pre-compiling exactly ? How to deal with syntax errors resulting in a failed load ? I'd also like to know more about the size of the mentioned problem. Whats the effect on the customer when module compilation is done on-the fly ? Whats the effect on the developer ? Klaus --- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
On Friday 14 September 2007 12:02, Klaus Kaempf wrote:
I've been wondering for a long time if we shouldn't seriously consider to handle the byte code the same way Perl does: Byte-compile "on the fly" and keep the byte code just in memory, never write it to disk. IMHO this would make handling YCP code _much_ easier.
What are the problems with the current way of pre-compiling exactly ?
Combining all the downsides of a compiler with most of the downsides of an interpreter. You have to explicitly byte-compile stuff you change. If you forget it, you are in trouble. And you might not even notice it. You can do changes all day long without any effect. For example, that makes inst-sys debugging even more painful than it already is. If you have to change a module, you have to remember to byte-compile all other modules (just the modules, not regular YCP files) that import it. And since there are a zillion dependencies between all of them, you have to do that in the right order. Otherwise you will have a lot of fun trying to debug upcoming problems.
How to deal with syntax errors resulting in a failed load ?
Just like we do with syntax errors with all other (non-module) YCP code. Having a pure interpreter does not obviate the need for proper development, for unit tests, for all kinds of tests you can do before the code hits the customer. Of course. But since we are really inconsistent with our handling of byte code, exactly what's the difference? I could understand that concern if we byte-compiled everything. But we don't. We have more checks (due to byte compilation) for some code (the modules) and less checks for other code. Both kinds of code will affect the customer: If any does not work due to syntax errors, the customer will get a show-stopper (one or some YaST apps won't work for him).
I'd also like to know more about the size of the mentioned problem.
Where have you been in the last couple of years? SCNR Building YaST2 is a pain. We all know that. Much of that pain comes from interdependencies of all the different parts of the code. We can't do very much about that for the C++ code, but it should really not hit us so badly for the YCP code, too. We've been throwing much of the advantage of having an interpreted language over board. Most of that is the result of having to byte-compile YCP code, and, worse, doing that in the correct build order. There are different approaches and different tools for that. Some developers simply hope they won't ever have to do it. That's OK for purely leaf packages. Other developers never leave their private sand box and simply never do "svn up" because when they do they are in big trouble. Some use tools like "y2makeall" which essentially recreate much of autobuild's internal logic, including carefully hand-crafted exception rules to manually throw out cyclic or broken (sic!) dependencies. I'd really like to go back to the days when handling YCP code was as simple as having the Makefile call "install" for each file, just copying to its target location. That was _so_ much simpler and _so_ much less error-prone.
Whats the effect on the customer when module compilation is done on-the fly?
That's what we have to find out, preferably in hard numbers, taken from real-life code.
Whats the effect on the developer ?
Less ulcers. CU -- Stefan Hundhammer <sh@suse.de> Penguin by conviction. YaST2 Development SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) Nürnberg, Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
On Friday 14 September 2007 12:23:39 Stefan Hundhammer wrote:
But since we are really inconsistent with our handling of byte code, exactly what's the difference? I could understand that concern if we byte-compiled everything. But we don't. We have more checks (due to byte compilation) for some code (the modules) and less checks for other code. Both kinds of code will affect the customer: If any does not work due to syntax errors, the customer will get a show-stopper (one or some YaST apps won't work for him).
I would prefer investing the time of making ycp compiler compile on the fly, on making existing interpreters work good with yast.
There are different approaches and different tools for that. Some developers simply hope they won't ever have to do it. That's OK for purely leaf packages. Other developers never leave their private sand box and simply never do "svn up" because when they do they are in big trouble.
There are projects much bigger than YaST, with much more complicated dependencies. People don't suffer the pain we do. What patterns I saw in projects like KDE codebase - policies for commiting changes on interfaces certain days only (ie: every monday the API breaks for trunk) - use of stuff like y2makeall (kdesvn-build).
Some use tools like "y2makeall" which essentially recreate much of autobuild's internal logic, including carefully hand-crafted exception rules to manually throw out cyclic or broken (sic!) dependencies.
I think allowing y2make all to link against in different locations is enough. For most YaST development you will use the system libraries. There are exceptions: - libzypp - dbus - hal - partitioning libs etc. Those should be recognized by y2makeall. Christian Kornacker is working on that. support for --libzypp-prefix will be added soon. Note, this also means y2makeall will build the complete YaST in a prefix. Duncan -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
Stefan Hundhammer wrote:
Combining all the downsides of a compiler with most of the downsides of an interpreter. You have to explicitly byte-compile stuff you change. If you forget it, you are in trouble. And you might not even notice it. You can do changes all day long without any effect.
For example, that makes inst-sys debugging even more painful than it already is.
If you have to change a module, you have to remember to byte-compile all other modules (just the modules, not regular YCP files) that import it. And since there are a zillion dependencies between all of them, you have to do that in the right order. Otherwise you will have a lot of fun trying to debug upcoming problems.
I've been debugging update inst-sys' YCP modules for a while and I can't say that it would be so hard. Even compiling every single that uses the changed one is not needed (unless you need the new API, of course :) )
How to deal with syntax errors resulting in a failed load ?
Just like we do with syntax errors with all other (non-module) YCP code.
Having a pure interpreter does not obviate the need for proper development, for unit tests, for all kinds of tests you can do before the code hits the customer. Of course.
Bytecode prevents from making more errors (customer focus), because when compiling it, ycpc checks all the syntax (in a RPM-build-time at least). And I additionally like the speedup from pre-compiled modules. Simple syntax check on a quite big script is just "too much" (that's the reason why I had to split some installation scripts into more smaller ones). Once we check the syntax in a build-time, we don't need to check it anymore and it returns hundred times for every single installed system.
But since we are really inconsistent with our handling of byte code, exactly what's the difference? I could understand that concern if we byte-compiled everything. But we don't. We have more checks (due to byte compilation) for some code (the modules) and less checks for other code. Both kinds of code will affect the customer: If any does not work due to syntax errors, the customer will get a show-stopper (one or some YaST apps won't work for him).
Calling 'make package' should check the syntax of clients as well.
Building YaST2 is a pain. We all know that. Much of that pain comes from interdependencies of all the different parts of the code. We can't do very much about that for the C++ code, but it should really not hit us so badly for the YCP code, too. We've been throwing much of the advantage of having an interpreted language over board. Most of that is the result of having to byte-compile YCP code, and, worse, doing that in the correct build order.
Well, yes, the correct build order. But you'd need the correct YCP modules anyway and some cross-dependencies or loop-imports might appear later.
There are different approaches and different tools for that. Some developers simply hope they won't ever have to do it. That's OK for purely leaf packages. Other developers never leave their private sand box and simply never do "svn up" because when they do they are in big trouble.
My 10.2 has a new 10.3's libzypp and yast2-core installed because I always need the latest one (when developing or testing installation features).
Some use tools like "y2makeall" which essentially recreate much of autobuild's internal logic, including carefully hand-crafted exception rules to manually throw out cyclic or broken (sic!) dependencies.
I hope that 'zypper --upgrade' (or whatever) could help here a lot. My solution is to keep my YaST (and related) packages up-to-date with /work (FACTORY) and time to time compile something needed from SVN.
I'd really like to go back to the days when handling YCP code was as simple as having the Makefile call "install" for each file, just copying to its target location. That was _so_ much simpler and _so_ much less error-prone.
Honestly, I don't remember that so I can't compare :)))
Whats the effect on the customer when module compilation is done on-the fly?
That's what we have to find out, preferably in hard numbers, taken from real-life code.
Slower YaST, more errors that appear during the runtime instead of the compile-time. That's also why I prefer writing modules in YCP to writing them in Perl. Anyway, this mail doesn't mean "I wanted to have the opposite opinion to HuHa's" :) I just wanted to add pros and cons I feel we should consider too. Bye Lukas
On 14/09/2007, Lukas Ocilka <lukas.ocilka@suse.cz> wrote:
Slower YaST, more errors that appear during the runtime instead of the compile-time. That's also why I prefer writing modules in YCP to writing them in Perl.
Hear hear, I tire of all the perl/python/ruby hype. Testing is not a substitute for verification. Sure YCP may be archaic & tiresome, but it's not as bad as the others. _ Benjamin Weber -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
* Benji Weber <b.weber@warwick.ac.uk> [Sep 14. 2007 15:16]:
On 14/09/2007, Lukas Ocilka <lukas.ocilka@suse.cz> wrote:
Slower YaST, more errors that appear during the runtime instead of the compile-time. That's also why I prefer writing modules in YCP to writing them in Perl.
Hear hear, I tire of all the perl/python/ruby hype. Testing is not a substitute for verification. Sure YCP may be archaic & tiresome, but it's not as bad as the others.
YCP was worse before we introduced static type checking. I agree with you that 'dynamically/loosely typed' languages are bad -- if one approaches them the traditional way where testing is considered optional bloat. My experience with e.g. Ruby is, that dynamic typing is really nice, provided(!) your testsuite has an almost 100% code coverage. Klaus --- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
* Stefan Hundhammer <sh@suse.de> [Sep 14. 2007 12:23]:
On Friday 14 September 2007 12:02, Klaus Kaempf wrote:
I'd also like to know more about the size of the mentioned problem.
Where have you been in the last couple of years? SCNR
:-) I've been on yast mailing lists where I found only very little traffic about this topic. The 'improve YaST developers environment' comes up every now and then but I do not have the impression that the 'pain' is strong enough.
Whats the effect on the developer ?
Less ulcers.
I'll pass FATE #2306 to you which explicitly requested improvements of the development environment. It now seems that there actually is interest in improvements. Klaus --- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
The 'improve YaST developers environment' comes up every now and then but I do not have the impression that the 'pain' is strong enough.
<nagging> Well then, it is because you've never been to Prague YaST office and never heard me screaming each time a change is commited into y2-core which invalidates all previously compiled bytecode and requires recompiling every single module I maintain. There is 5 of them, each one has at least one dependency, so I need to recompile 5+ modules to finally get down to real work. Or better (and that happens even more often), if a change is commited to libzypp, forcing me (after building libzypp) to recompile y2-ncurses (because I maintain it) and y2-qt, together with y2-pkg-bindings and y2-perl-bindings. It's a nightmare and it's how real pain looks like, and I'm not saying that just because I'm an overstressed female ... </nagging> -- \\\\\ Katarina Machalkova \\\\\\\__o YaST developer __\\\\\\\'/_ & hedgehog painter
* Katarina Machalkova <kmachalkova@suse.cz> [Sep 14. 2007 14:08]:
The 'improve YaST developers environment' comes up every now and then but I do not have the impression that the 'pain' is strong enough.
<nagging> Well then, it is because you've never been to Prague YaST office and never heard me screaming each time a change is commited into y2-core which invalidates all previously compiled bytecode and requires recompiling every single module I maintain.
I want to encourage you and others to come up with a proposal on what to improve in order to ease the pain. Klaus --- SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
On Friday 14 September 2007 14:24, Klaus Kaempf wrote:
Well then, it is because you've never been to Prague YaST office and never heard me screaming each time a change is commited into y2-core which invalidates all previously compiled bytecode and requires recompiling every single module I maintain.
I want to encourage you and others to come up with a proposal on what to improve in order to ease the pain.
---------- Forwarded Message ---------- Subject: Re: [yast-devel] how to mark *.ybc file outdated? Date: Friday 14 September 2007 11:51 From: Stefan Hundhammer <sh@suse.de> To: yast-devel@opensuse.org On Friday 14 September 2007 09:52, Johannes Meixner wrote: [...byte code files outdated...] I've been wondering for a long time if we shouldn't seriously consider to handle the byte code the same way Perl does: Byte-compile "on the fly" and keep the byte code just in memory, never write it to disk. IMHO this would make handling YCP code _much_ easier. ------------------------------------------------------- CU -- Stefan Hundhammer <sh@suse.de> Penguin by conviction. YaST2 Development SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) Nürnberg, Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
On Friday 14 September 2007 05:02, Klaus Kaempf wrote:
I'd also like to know more about the size of the mentioned problem. Whats the effect on the customer when module compilation is done on-the fly? Whats the effect on the developer ?
From user prospective: Printer module on first start gives message "printer database is changed" (or something similar), than it is doing something for long enough (Athlon 64 3500) to bring question what is changed in database in new installed system? Why database is not created before? If that would be repeated on every module start, than it will be a problem to think how to optimize it. If that is time necessary to compile .ycp to .ycb it is way too long to repeat every time. -- Regards, Rajko. -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
On Friday 14 September 2007 13:03, Rajko M. wrote:
On Friday 14 September 2007 05:02, Klaus Kaempf wrote:
I'd also like to know more about the size of the mentioned problem. Whats the effect on the customer when module compilation is done on-the fly? Whats the effect on the developer ?
From user prospective: Printer module on first start gives message "printer database is changed" (or something similar), than it is doing something for long enough (Athlon 64 3500) to bring question what is changed in database in new installed system? Why database is not created before? If that would be repeated on every module start, than it will be a problem to think how to optimize it.
If that is time necessary to compile .ycp to .ycb it is way too long to repeat every time.
This is utterly unrelated. Rebuilding the printer database has nothing whatsoever to do with byte-compiling YCP code. CU -- Stefan Hundhammer <sh@suse.de> Penguin by conviction. YaST2 Development SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg) Nürnberg, Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
On Monday 17 September 2007 07:00:34 am Stefan Hundhammer wrote:
On Friday 14 September 2007 13:03, Rajko M. wrote:
On Friday 14 September 2007 05:02, Klaus Kaempf wrote:
I'd also like to know more about the size of the mentioned problem. Whats the effect on the customer when module compilation is done on-the fly? Whats the effect on the developer ?
From user prospective: Printer module on first start gives message "printer database is changed" (or something similar), than it is doing something for long enough (Athlon 64 3500) to bring question what is changed in database in new installed system? Why database is not created before? If that would be repeated on every module start, than it will be a problem to think how to optimize it.
If that is time necessary to compile .ycp to .ycb it is way too long to repeat every time.
This is utterly unrelated. Rebuilding the printer database has nothing whatsoever to do with byte-compiling YCP code.
That is why I put it under "if". Thank you for the answer Stefan. -- Regards, Rajko. -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
Dne Friday 14 of September 2007 13:03:11 Rajko M. napsal(a):
On Friday 14 September 2007 05:02, Klaus Kaempf wrote:
I'd also like to know more about the size of the mentioned problem. Whats the effect on the customer when module compilation is done on-the fly? Whats the effect on the developer ?
From user prospective: Printer module on first start gives message "printer database is changed" (or something similar), than it is doing something for long enough (Athlon 64 3500) to bring question what is changed in database in new installed system? Why database is not created before? If that would be repeated on every module start, than it will be a problem to think how to optimize it.
The problem is that there are several packages holding the drivers, which have to be compiled into one database. That's why there can not be one database which can be used out of the box after installation. Jiri -- Regards, Jiri Srain YaST Team Leader --------------------------------------------------------------------- SUSE LINUX, s.r.o. e-mail: jsrain@suse.cz Lihovarska 1060/12 tel: +420 284 028 959 190 00 Praha 9 fax: +420 284 028 951 Czech Republic http://www.suse.cz -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org For additional commands, e-mail: yast-devel+help@opensuse.org
participants (10)
-
Benji Weber -
Duncan Mac-Vicar Prett -
Jiri Srain -
Johannes Meixner -
Katarina Machalkova -
Klaus Kaempf -
Lukas Ocilka -
Rajko M. -
Stanislav Visnovsky -
Stefan Hundhammer