[yast-devel] Executing our own code: inst-sys clients
While working on a bug related to the partitions_proposal client of y-storage-ng, I wondered why we don't have a simple way to call our own code. I want to be able to start the stuff that I work on in my normal work environment. I was told that "it just takes 4-5 lines of Ruby code", but that's not a useful answer; why don't we have a ready-made wrapper for those things if it's really so simple? I have no clue what I need to set up first before I can call that thing programatically. I need to see it in connection with the proposal because that's what it's used for. We have a lot of infrastructure already in place: - For storage scenarios, we have YAML files describing the existing disk setup; we use them in the unit tests. - We have control XML files for all our products. - We have a lot of AutoYaST files that we use for automated tests. - We have an examples/ subdirectory in y-storage-ng, but that doesn't contain much. It's a PITA to start a real installation and then modify the inst-sys with bind-mounts and whatnot, always copying files to that inst-sys after each change. The easier we make it to call our code directly, the more it will be done which means much more thorough (manual!) testing, and the easier it will be to develop on that code. Automated testing can only go so far; unit tests use a lot of mocking and assumptions, and OpenQA is limited in the number of scenarios that can reasonably be tested, and it isn't helpful for us during development: I need to see what I am doing, I can't wait for half a week for a test result, and I don't want to check in code that I never saw running (which for Ruby typically means that there will be one crash for every 3 lines of code written). So, for simple things I propose we get into the habit of providing a simple example that can easily be called. Back when I wrote the UI, I did that for every single widget, and even in different degrees of difficulty and complexity: PushButton1.rb, PushButton2.rb, ... This encourages people to execute the code, to see how it behaves, how it is supposed to be used, and to do some first level of testing. I don't know about you guys, but I cannot develop without that. Rspec and unit testing have limits, even more so with the arcane error messages that Rspec throws at you when you do anything wrong. How can we get there? Don't you have the same problems? How do you develop code that will live in the inst-sys? I suggest we set up a Jangouts call to discuss this. Kind regards -- Stefan Hundhammer <shundhammer@suse.de> YaST Developer SUSE Linux GmbH GF: Felix Imendörffer, Jane Smithard, Graham Norton; HRB 21284 (AG Nürnberg) Maxfeldstr. 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
V Wed, 15 Aug 2018 15:47:03 +0200 Stefan Hundhammer <shundhammer@suse.de> napsáno:
While working on a bug related to the partitions_proposal client of y-storage-ng, I wondered why we don't have a simple way to call our own code.
I want to be able to start the stuff that I work on in my normal work environment.
I was told that "it just takes 4-5 lines of Ruby code", but that's not a useful answer; why don't we have a ready-made wrapper for those things if it's really so simple? I have no clue what I need to set up first before I can call that thing programatically. I need to see it in connection with the proposal because that's what it's used for.
We have a lot of infrastructure already in place:
- For storage scenarios, we have YAML files describing the existing disk setup; we use them in the unit tests.
- We have control XML files for all our products.
- We have a lot of AutoYaST files that we use for automated tests.
- We have an examples/ subdirectory in y-storage-ng, but that doesn't contain much.
It's a PITA to start a real installation and then modify the inst-sys with bind-mounts and whatnot, always copying files to that inst-sys after each change.
The easier we make it to call our code directly, the more it will be done which means much more thorough (manual!) testing, and the easier it will be to develop on that code.
Automated testing can only go so far; unit tests use a lot of mocking and assumptions, and OpenQA is limited in the number of scenarios that can reasonably be tested, and it isn't helpful for us during development: I need to see what I am doing, I can't wait for half a week for a test result, and I don't want to check in code that I never saw running (which for Ruby typically means that there will be one crash for every 3 lines of code written).
So, for simple things I propose we get into the habit of providing a simple example that can easily be called. Back when I wrote the UI, I did that for every single widget, and even in different degrees of difficulty and complexity: PushButton1.rb, PushButton2.rb, ...
Hi HuHa, I think you open interesting topic as insts-sys is a bit limited and sometimes set it up is not trivial. So let me try to answer some of your question ( not sure if it is what you want to read :) Problem with examples is that it stop working and no-one notice it. So for me examples is great if it comes with automatic validation. But I agree that for UI it makes a lot of sense to verify that it looks as it should.
This encourages people to execute the code, to see how it behaves, how it is supposed to be used, and to do some first level of testing. I don't know about you guys, but I cannot develop without that. Rspec and unit testing have limits, even more so with the arcane error messages that Rspec throws at you when you do anything wrong.
Well, for me it really depends which part of code I develop. If it is backend, public API or something low level then rspec is great. For UI I prefer examples which I can manually run. This is e.g. reason why for expert partitioner we had (or still have ) special client that can work with passed yaml file.
How can we get there? Don't you have the same problems? How do you develop code that will live in the inst-sys?
well, integrated byebug debugger helps. But my usual workflow is to write code, write tests that verify it works as expected. Then I call rake osc:build and via dud I try to run installation with modified code to be sure I do not overlook anything. I do not say it is perfect, but usually works fine for me, just it can be faster. What I did in past was also usage of /y2update, but I often forget to copy back my manual modification of installation sources.
I suggest we set up a Jangouts call to discuss this.
make sense for me. just finding good time will be hard. Josef
Kind regards
-- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 15.08.2018 16:08, Josef Reidinger wrote:
Problem with examples is that it stop working and no-one notice it.
This is a myth that I debunked several times already.
Well, for me it really depends which part of code I develop. If it is backend, public API or something low level then rspec is great.
With the amount of mocking we typically do in our unit tests, this doesn't really help very much. That's my point. It only touches the surface.
For UI I prefer examples which I can manually run. This is e.g. reason why for expert partitioner we had (or still have ) special client that can work with passed yaml file.
The expert partitioner can run standalone; that one is not a problem. All the stuff that is used directly or indirectly by the proposal is a big problem.
How can we get there? Don't you have the same problems? How do you develop code that will live in the inst-sys? well, integrated byebug debugger helps.
Sorry, no. That byebug debugger is worse than useless. It pretends to offer something that it doesn't deliver in real life. This is like trying to debug a complex KDE application with plain gdb - an exercise in futility.
But my usual workflow is to write code, write tests that verify it works as expected. Then I call rake osc:build and via dud I try to run installation with modified code to be sure I do not overlook anything. I do not say it is perfect, but usually works fine for me, just it can be faster. What I did in past was also usage of /y2update, but I often forget to copy back my manual modification of installation sources.
Yes, that's what I meant with PITA. I could not imagine anything more complicated and error-prone and less productive. This is my point. Kind regards -- Stefan Hundhammer <shundhammer@suse.de> YaST Developer SUSE Linux GmbH GF: Felix Imendörffer, Jane Smithard, Graham Norton; HRB 21284 (AG Nürnberg) Maxfeldstr. 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
V Wed, 15 Aug 2018 16:38:02 +0200 Stefan Hundhammer <shundhammer@suse.de> napsáno:
On 15.08.2018 16:08, Josef Reidinger wrote:
Problem with examples is that it stop working and no-one notice it.
This is a myth that I debunked several times already.
Well, for me it really depends which part of code I develop. If it is backend, public API or something low level then rspec is great.
With the amount of mocking we typically do in our unit tests, this doesn't really help very much. That's my point. It only touches the surface.
So maybe we should discuss what is reasonable amount of mocking. I think similar topic was also described in article that Arvin send about testing.
For UI I prefer examples which I can manually run. This is e.g. reason why for expert partitioner we had (or still have ) special client that can work with passed yaml file.
The expert partitioner can run standalone; that one is not a problem. All the stuff that is used directly or indirectly by the proposal is a big problem.
So maybe we need to identify why proposal is so big problem? If we can somehow base proposal on some format that describe conditions on which proposal is created?
How can we get there? Don't you have the same problems? How do you develop code that will live in the inst-sys? well, integrated byebug debugger helps.
Sorry, no. That byebug debugger is worse than useless. It pretends to offer something that it doesn't deliver in real life. This is like trying to debug a complex KDE application with plain gdb - an exercise in futility.
It probably depends what exactly you want to debug. As you can read with it various internal variables and try to e.g. call propose again to check how it behave when you modify some condition. But I agree that it is not high level tool that will allow you to easily modify some installation condition.
But my usual workflow is to write code, write tests that verify it works as expected. Then I call rake osc:build and via dud I try to run installation with modified code to be sure I do not overlook anything. I do not say it is perfect, but usually works fine for me, just it can be faster. What I did in past was also usage of /y2update, but I often forget to copy back my manual modification of installation sources.
Yes, that's what I meant with PITA. I could not imagine anything more complicated and error-prone and less productive. This is my point.
well, my current workflow with osc:build and dud is exactly what customer doing in L3 bugs with PTFs, so I would not call it error-prone and complicated. At least I am not aware about anything better otherwise I believe we will use it also on customer side.
Kind regards
-- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 15.08.2018 17:15, Josef Reidinger wrote: ... ... This is all beyond the point, and this is why I had said beforehand that it's pointless to discuss anything meaningful on this mailing list. It drifts off to sidelines every single time. The topic always remains untouched, questions always remain unanswered. Giving up. -- Stefan Hundhammer <shundhammer@suse.de> YaST Developer SUSE Linux GmbH GF: Felix Imendörffer, Jane Smithard, Graham Norton; HRB 21284 (AG Nürnberg) Maxfeldstr. 5, 90409 Nürnberg, Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
V Wed, 15 Aug 2018 17:26:24 +0200 Stefan Hundhammer <shundhammer@suse.de> napsáno:
On 15.08.2018 17:15, Josef Reidinger wrote: ... ...
This is all beyond the point, and this is why I had said beforehand that it's pointless to discuss anything meaningful on this mailing list. It drifts off to sidelines every single time. The topic always remains untouched, questions always remain unanswered.
Giving up.
OK, so lets summarize your questions and answers I write: - How can we get there? as I write try to arrange call for it ( with good time slot ). Maybe it can be topic for retrospective, because if you want it everywhere then it have to be part of acceptance criteria and it also slow down development for people that use different way for manual run in insts-sys ( maybe in the end they benefit from it ). - Don't you have the same problems? I write some problems I had in past and as a bonus I can say that for me the biggest problem is insts-sys on strange architectures for which is hard to get machine and if you get it then it use some obscure communication tool. - How do you develop code that will live in the inst-sys? I write how I do it. I believe others have different way and can write their own way. I also try to explain why I use my own way (e.g. same way which use customer for PTFs). If there was more questions, then I overlook it, but I hope I answer all your questions. Josef -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 08/15/2018 05:15 PM, Josef Reidinger wrote:
V Wed, 15 Aug 2018 16:38:02 +0200 Stefan Hundhammer <shundhammer@suse.de> napsáno:
On 15.08.2018 16:08, Josef Reidinger wrote:
Problem with examples is that it stop working and no-one notice it.
For UI I prefer examples which I can manually run. This is e.g. reason why for expert partitioner we had (or still have ) special client that can work with passed yaml file.
The expert partitioner can run standalone; that one is not a problem. All the stuff that is used directly or indirectly by the proposal is a big problem.
So maybe we need to identify why proposal is so big problem? If we can somehow base proposal on some format that describe conditions on which proposal is created?
I'm a little bit lost here. What are we missing exactly? We already have tests that check how the proposal work for a given initial disks layout and a given configuration from control.xml (or role, or whatever). We also have src/examples/proposal_settings_dialog.rb that takes a control file and a storage-ng scenario as arguments and present the dialog, printing the proposal at the end. Of course, we could have exactly the same but with the previous dialog (the one that contains the buttons for "Guided Setup" and "Expert Partitioner"). So it's perfectly possible (and we do it all the time) to run the proposal or any of its components dialogs based on a description of the conditions (the control.xml and the devicegraph). Or did I miss the point? What is missing in that regard? Cheers. -- Ancor González Sosa YaST Team at SUSE Linux GmbH -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 08/15/2018 04:38 PM, Stefan Hundhammer wrote:
On 15.08.2018 16:08, Josef Reidinger wrote:
How can we get there? Don't you have the same problems? How do you develop code that will live in the inst-sys? well, integrated byebug debugger helps.
Sorry, no. That byebug debugger is worse than useless. It pretends to offer something that it doesn't deliver in real life. This is like trying to debug a complex KDE application with plain gdb - an exercise in futility.
What exactly do you miss? As said in another mail, after writing a script/test to reproduce the problem, I find dropping a couple of calls to byebug here and there to be EXTREMELY useful. What do you miss from other debuggers? Just curious. I want to know what kind of debugging tools you are missing. Maybe there is something similar for Ruby. I have not researched beyond byebug... maybe because I'm just too easy to please. ;-) For example, in the past I used some visual frontends for gdb when writing C/C++, like ddd[1] and so on. But since in Ruby I don't have to manage the pointers and the memory myself I have not felt the need of them anymore. Cheers. [1] https://www.gnu.org/software/ddd/ -- Ancor González Sosa YaST Team at SUSE Linux GmbH -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 08/15/2018 03:47 PM, Stefan Hundhammer wrote:
While working on a bug related to the partitions_proposal client of y-storage-ng, I wondered why we don't have a simple way to call our own code.
I want to be able to start the stuff that I work on in my normal work environment.
I was told that "it just takes 4-5 lines of Ruby code", but that's not a useful answer; why don't we have a ready-made wrapper for those things if it's really so simple?
I guess because the exact 4-5 lines depend of what you want to run/test. With modern code, hopefully in all cases running a component (a proposal client, an installation step, a dialog, etc.) is just a matter of instantiating an object of the corresponding class (the dialog, the client, whatever) and running the offending method (typically #run for clients and dialogs). So the basis is always: ``` require "yast" require "y2whatever/dialogs/foo" # Assuming the constructor needs no arguments dialog = Y2Whatever::Dialogs::Foo.new dialog.run ``` There is nothing special (or super useful) there.
I have no clue what I need to set up first before I can call that thing programatically.
And that's the real problem... which I don't see how to solve in a generic way. In order to debug a concrete bug, maybe Dialogs::Foo needs some other YaST module to be previously initialized with some specific value. Or it needs to receive a particular value as constructor argument... It really depends on what you need to reproduce and what Foo expects.
I need to see it in connection with the proposal because that's what it's used for.
We have a lot of infrastructure already in place:
- For storage scenarios, we have YAML files describing the existing disk setup; we use them in the unit tests.
- We have control XML files for all our products.
- We have a lot of AutoYaST files that we use for automated tests.
- We have an examples/ subdirectory in y-storage-ng, but that doesn't contain much.
It's a PITA to start a real installation and then modify the inst-sys with bind-mounts and whatnot, always copying files to that inst-sys after each change.
The easier we make it to call our code directly, the more it will be done which means much more thorough (manual!) testing, and the easier it will be to develop on that code.
Fully agreed. I just do the inst-sys thingy at the end, to verify. For development I usually start writing a RSpec test to reproduce the situation or write a script similar to the one at the start of this mail... and then I use a lot byebug. :-)
Automated testing can only go so far; unit tests use a lot of mocking and assumptions, and OpenQA is limited in the number of scenarios that can reasonably be tested, and it isn't helpful for us during development: I need to see what I am doing, I can't wait for half a week for a test result, and I don't want to check in code that I never saw running (which for Ruby typically means that there will be one crash for every 3 lines of code written).
So, for simple things I propose we get into the habit of providing a simple example that can easily be called. Back when I wrote the UI, I did that for every single widget, and even in different degrees of difficulty and complexity: PushButton1.rb, PushButton2.rb, ...
This encourages people to execute the code, to see how it behaves, how it is supposed to be used, and to do some first level of testing. I don't know about you guys, but I cannot develop without that. Rspec and unit testing have limits, even more so with the arcane error messages that Rspec throws at you when you do anything wrong.
That's what I use RSpec for. To document how the code is supposed to be used and, for the same price, to make sure the code is indeed executed on every submission (not only encouraged to be executed for others reading the example). For me, reading the Yardoc documentation and the RSpec tests is usually enough to make my mind and jump into development. So, somehow, for me the RSpec tests play the role of those examples. They contain the four (or more) lines I need to initialize the class in several concrete situations, a good overview of the supported scenarios... Everything I would need as an example and hopefully up to date (since they are actually executed in a regular basis, unlike the example scripts).
How can we get there? Don't you have the same problems? How do you develop code that will live in the inst-sys?
TBH, I don't feel I have the same problem. A said, it usually it takes me some minutes to reproduce the problem in a lightweight testing environment (a RSpec test or an example script) using Yardoc and RSpec as source of information for that. Of course, I still do the DUD thingy at the end (after fixing the stuff in that testing environment) to make sure everything works for real... and sometimes to find out further problems. :-(
I suggest we set up a Jangouts call to discuss this.
Fine for me. Any proposal? Cheers. -- Ancor González Sosa YaST Team at SUSE Linux GmbH -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
participants (3)
-
Ancor Gonzalez Sosa -
Josef Reidinger -
Stefan Hundhammer