[yast-devel] 2nd attempt: do we need a layer between libstorage-ng and (the rest) of YaST?
Since the original thread is already ruined with discussions about concrete API design details that were not the main topic of my question, I will try once again from scratch. Do we need a layer between libstorage-ng and (the rest) of YaST? I think so. Why do I? TLDR; libstorage-ng is multilanguage (with a Ruby-agnostic API, to say the least) and YaST-agnostic. Therefore, we need a layer to close the distance between libstorage-ng and Ruby/YaST. The alternative is to have boilerplate code all along YaST (like downcast) and our own customization for the Ruby tools. If you agree, please say so. If you disagree, please provide counterarguments. Now the long version of my reasoning (and you know that "long" means in my case). ;-) A) Libstorage-ng API is designed to be useful for many languages and for many potential users, not only Ruby and YaST. B) Libstorage-ng is designed to be relatively low-level, not hiding details about how the different moving pieces (devices, filesystems, etc.) are connected and allowing to model all the possibilities that can happen in reality. That's perfectly fine. It makes libstorage-ng more versatile and powerful. But it has some implications. A consequence for (A): when taking API design decisions, if there are several ways of doing something and no way is universally better than the others[*], libstorage-ng always work in the way that is not standard in Ruby. That means libstorage-ng does not play nicely with the Ruby ecosystem, causing extra work for the Ruby developers just to keep using the tools chosen for YaST development and testing. For (B) it means libstorage-ng does not offer a direct answer to the kind of questions usually asked all around YaST (yast-packager, yast-bootloader, yast-country....). Although examples-based discussion has proved to lead to off-topic, I will try once again: A typical question from YaST to the storage layer: - For this given filesystem, which are the physical disks (whatever technology is used)? That question would be plain wrong from the libstorage-ng POV, because there is no such a direct relationship (not always) and therefore the user of the library is expected to know all the possible paths from a a filesystem to its disks and traverse all those paths doing the corresponding checks for empty and performing the downcasts in every step. In other words, there is no a_filesystem.all_disks But the user is expected to do this blk_dev = a_filesystem.blk_devices[0] if Storage.encryption?(blk_device) blk_device = Storage.to_encryption(blk_device).blk_device end return [Storage.to_disk(blk_dev)] if Storage.disk?(blk_dev) if Storage.partition?(blk_device) partition = Storage.to_partition(blk_device) if Storage.disk?(partition.partition_table.partitionable) return [Storage.to_disk(partition.partition_table.partitionable)] else # MD case end elsif Storage.lvm_lv?(blk_device) # The LVM case, which is double the size than the partition one end This is a simplified example don't taking into account he possibilities of multi-device filesystems or encryptions in other layers than the very first one (right below the filesystem). While adapting other modules to use the new libstorage directly we have found many examples of that gap between the abstraction level expected by YaST and the level of "clarity" about the internal structure enforced by libstorage-ng. So please refrain from discussing every single variable name in the example or providing an alternative with 4 fewer lines. That will not change the main point. :) So, as said at the beginning, libstorage-ng is perfect as it is, for the goal it is designed. That goal is not to be Ruby-compliant or to match the YaST use-case 100%. So we need an extra layer to walk that way. IMHO, opposing to the creation of that layer just to avoid the fact that some developers has to know and work with both layers is not reasonable. If you disagree, please provide counter-arguments to the provided arguments, instead of discussing the wording of the examples. ;-) Cheers. [*] As proved by the endless discussions focused on examples and counter-examples in which nobody convinces nobody else. -- Ancor González Sosa YaST Team at SUSE Linux GmbH -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
In short I agree that layer between would be nice. And I think in past such layer is provided by y2-storage and I think it make sense to continue with it. Only reason to not place it directly to y2-storage is if someone else would like to use storage-ng from ruby, then using better API from y2-storage can be problematic. Josef On Mon, 27 Feb 2017 08:51:01 +0100 Ancor Gonzalez Sosa <ancor@suse.de> wrote:
Since the original thread is already ruined with discussions about concrete API design details that were not the main topic of my question, I will try once again from scratch.
Do we need a layer between libstorage-ng and (the rest) of YaST?
I think so. Why do I?
TLDR; libstorage-ng is multilanguage (with a Ruby-agnostic API, to say the least) and YaST-agnostic. Therefore, we need a layer to close the distance between libstorage-ng and Ruby/YaST. The alternative is to have boilerplate code all along YaST (like downcast) and our own customization for the Ruby tools.
If you agree, please say so.
If you disagree, please provide counterarguments.
Now the long version of my reasoning (and you know that "long" means in my case). ;-)
A) Libstorage-ng API is designed to be useful for many languages and for many potential users, not only Ruby and YaST.
B) Libstorage-ng is designed to be relatively low-level, not hiding details about how the different moving pieces (devices, filesystems, etc.) are connected and allowing to model all the possibilities that can happen in reality.
That's perfectly fine. It makes libstorage-ng more versatile and powerful. But it has some implications.
A consequence for (A): when taking API design decisions, if there are several ways of doing something and no way is universally better than the others[*], libstorage-ng always work in the way that is not standard in Ruby.
That means libstorage-ng does not play nicely with the Ruby ecosystem, causing extra work for the Ruby developers just to keep using the tools chosen for YaST development and testing.
For (B) it means libstorage-ng does not offer a direct answer to the kind of questions usually asked all around YaST (yast-packager, yast-bootloader, yast-country....). Although examples-based discussion has proved to lead to off-topic, I will try once again:
A typical question from YaST to the storage layer: - For this given filesystem, which are the physical disks (whatever technology is used)?
That question would be plain wrong from the libstorage-ng POV, because there is no such a direct relationship (not always) and therefore the user of the library is expected to know all the possible paths from a a filesystem to its disks and traverse all those paths doing the corresponding checks for empty and performing the downcasts in every step.
In other words, there is no
a_filesystem.all_disks
But the user is expected to do this
blk_dev = a_filesystem.blk_devices[0] if Storage.encryption?(blk_device) blk_device = Storage.to_encryption(blk_device).blk_device end return [Storage.to_disk(blk_dev)] if Storage.disk?(blk_dev) if Storage.partition?(blk_device) partition = Storage.to_partition(blk_device) if Storage.disk?(partition.partition_table.partitionable) return [Storage.to_disk(partition.partition_table.partitionable)] else # MD case end elsif Storage.lvm_lv?(blk_device) # The LVM case, which is double the size than the partition one end
This is a simplified example don't taking into account he possibilities of multi-device filesystems or encryptions in other layers than the very first one (right below the filesystem).
While adapting other modules to use the new libstorage directly we have found many examples of that gap between the abstraction level expected by YaST and the level of "clarity" about the internal structure enforced by libstorage-ng. So please refrain from discussing every single variable name in the example or providing an alternative with 4 fewer lines. That will not change the main point. :)
So, as said at the beginning, libstorage-ng is perfect as it is, for the goal it is designed. That goal is not to be Ruby-compliant or to match the YaST use-case 100%. So we need an extra layer to walk that way.
IMHO, opposing to the creation of that layer just to avoid the fact that some developers has to know and work with both layers is not reasonable.
If you disagree, please provide counter-arguments to the provided arguments, instead of discussing the wording of the examples. ;-)
Cheers.
[*] As proved by the endless discussions focused on examples and counter-examples in which nobody convinces nobody else.
-- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 02/27/2017 09:14 AM, Josef Reidinger wrote:
In short I agree that layer between would be nice. And I think in past such layer is provided by y2-storage and I think it make sense to continue with it.
And being honest to ourselves, that layer is also there in yast-storage-ng, just in a non-explicit way. We have refinements and patches on top of the libstorage-ng classes to fulfill the YaST use-cases and requirements. Using those refinements has already saved us from adapting the code at yast-country, yast-packager, yast-bootloader and the storage proposal when we introduced encryption support in the library. Patches and refinements are an un-obvious solution (patches are even dangerous). The explicit solution (a visible middle layer) is much preferred in my opinion.
Only reason to not place it directly to y2-storage is if someone else would like to use storage-ng from ruby, then using better API from y2-storage can be problematic.
Not sure if I got this point.
Josef
Cheers.
On Mon, 27 Feb 2017 08:51:01 +0100 Ancor Gonzalez Sosa <ancor@suse.de> wrote:
Since the original thread is already ruined with discussions about concrete API design details that were not the main topic of my question, I will try once again from scratch.
Do we need a layer between libstorage-ng and (the rest) of YaST?
I think so. Why do I?
TLDR; libstorage-ng is multilanguage (with a Ruby-agnostic API, to say the least) and YaST-agnostic. Therefore, we need a layer to close the distance between libstorage-ng and Ruby/YaST. The alternative is to have boilerplate code all along YaST (like downcast) and our own customization for the Ruby tools.
If you agree, please say so.
If you disagree, please provide counterarguments.
Now the long version of my reasoning (and you know that "long" means in my case). ;-)
A) Libstorage-ng API is designed to be useful for many languages and for many potential users, not only Ruby and YaST.
B) Libstorage-ng is designed to be relatively low-level, not hiding details about how the different moving pieces (devices, filesystems, etc.) are connected and allowing to model all the possibilities that can happen in reality.
That's perfectly fine. It makes libstorage-ng more versatile and powerful. But it has some implications.
A consequence for (A): when taking API design decisions, if there are several ways of doing something and no way is universally better than the others[*], libstorage-ng always work in the way that is not standard in Ruby.
That means libstorage-ng does not play nicely with the Ruby ecosystem, causing extra work for the Ruby developers just to keep using the tools chosen for YaST development and testing.
For (B) it means libstorage-ng does not offer a direct answer to the kind of questions usually asked all around YaST (yast-packager, yast-bootloader, yast-country....). Although examples-based discussion has proved to lead to off-topic, I will try once again:
A typical question from YaST to the storage layer: - For this given filesystem, which are the physical disks (whatever technology is used)?
That question would be plain wrong from the libstorage-ng POV, because there is no such a direct relationship (not always) and therefore the user of the library is expected to know all the possible paths from a a filesystem to its disks and traverse all those paths doing the corresponding checks for empty and performing the downcasts in every step.
In other words, there is no
a_filesystem.all_disks
But the user is expected to do this
blk_dev = a_filesystem.blk_devices[0] if Storage.encryption?(blk_device) blk_device = Storage.to_encryption(blk_device).blk_device end return [Storage.to_disk(blk_dev)] if Storage.disk?(blk_dev) if Storage.partition?(blk_device) partition = Storage.to_partition(blk_device) if Storage.disk?(partition.partition_table.partitionable) return [Storage.to_disk(partition.partition_table.partitionable)] else # MD case end elsif Storage.lvm_lv?(blk_device) # The LVM case, which is double the size than the partition one end
This is a simplified example don't taking into account he possibilities of multi-device filesystems or encryptions in other layers than the very first one (right below the filesystem).
While adapting other modules to use the new libstorage directly we have found many examples of that gap between the abstraction level expected by YaST and the level of "clarity" about the internal structure enforced by libstorage-ng. So please refrain from discussing every single variable name in the example or providing an alternative with 4 fewer lines. That will not change the main point. :)
So, as said at the beginning, libstorage-ng is perfect as it is, for the goal it is designed. That goal is not to be Ruby-compliant or to match the YaST use-case 100%. So we need an extra layer to walk that way.
IMHO, opposing to the creation of that layer just to avoid the fact that some developers has to know and work with both layers is not reasonable.
If you disagree, please provide counter-arguments to the provided arguments, instead of discussing the wording of the examples. ;-)
Cheers.
[*] As proved by the endless discussions focused on examples and counter-examples in which nobody convinces nobody else.
-- Ancor González Sosa YaST Team at SUSE Linux GmbH -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 02/27/2017 07:51 AM, Ancor Gonzalez Sosa wrote:
Since the original thread is already ruined with discussions about concrete API design details that were not the main topic of my question, I will try once again from scratch.
Do we need a layer between libstorage-ng and (the rest) of YaST?
I think so. Why do I?
TLDR; libstorage-ng is multilanguage (with a Ruby-agnostic API, to say the least) and YaST-agnostic. Therefore, we need a layer to close the distance between libstorage-ng and Ruby/YaST. The alternative is to have boilerplate code all along YaST (like downcast) and our own customization for the Ruby tools.
If you agree, please say so.
As you already presented your arguments, all I'm going to say is that I fully agree. Thanks! -- Imobach González Sosa YaST team at SUSE LINUX GmbH Blog: https://imobachgs.github.io/ Twitter: @imobachgs -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Mon, Feb 27, 2017 at 08:51:01AM +0100, Ancor Gonzalez Sosa wrote:
Since the original thread is already ruined with discussions about concrete API design details that were not the main topic of my question, I will try once again from scratch.
Sorry to bother you, but if you continue to bring such bad examples this thread will also be ruined by the end of the day.
a_filesystem.all_disks
But the user is expected to do this
blk_dev = a_filesystem.blk_devices[0] if Storage.encryption?(blk_device) blk_device = Storage.to_encryption(blk_device).blk_device end return [Storage.to_disk(blk_dev)] if Storage.disk?(blk_dev) if Storage.partition?(blk_device) partition = Storage.to_partition(blk_device) if Storage.disk?(partition.partition_table.partitionable) return [Storage.to_disk(partition.partition_table.partitionable)] else # MD case end elsif Storage.lvm_lv?(blk_device) # The LVM case, which is double the size than the partition one end
Or just: for filesystem.ancestors do |device| do ret = [] ret << device if disk?(device) return ret done ciao Arvin -- Arvin Schnell, <aschnell@suse.com> Senior Software Engineer, Research & Development SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 02/27/2017 10:21 AM, Arvin Schnell wrote:
On Mon, Feb 27, 2017 at 08:51:01AM +0100, Ancor Gonzalez Sosa wrote:
Since the original thread is already ruined with discussions about concrete API design details that were not the main topic of my question, I will try once again from scratch.
Sorry to bother you, but if you continue to bring such bad examples this thread will also be ruined by the end of the day.
a_filesystem.all_disks
But the user is expected to do this
blk_dev = a_filesystem.blk_devices[0] if Storage.encryption?(blk_device) blk_device = Storage.to_encryption(blk_device).blk_device end return [Storage.to_disk(blk_dev)] if Storage.disk?(blk_dev) if Storage.partition?(blk_device) partition = Storage.to_partition(blk_device) if Storage.disk?(partition.partition_table.partitionable) return [Storage.to_disk(partition.partition_table.partitionable)] else # MD case end elsif Storage.lvm_lv?(blk_device) # The LVM case, which is double the size than the partition one end
Or just:
for filesystem.ancestors do |device| do ret = [] ret << device if disk?(device) return ret done
Ok, I'll bite the hook. That returns objects of class Device, not disks. You forgot the downcast. So it would be ret << Storage.to_disk(device) if disk?(device) And as I have tried to explain (and apparently failed), checking for the real type of something and then performing a downcast is already non-ruby. But let's try another less simplistic example, in which I expect at least some downcasts in the bare libstorage-ng version: For a given disk, an array of its encrypted logical partitions containing an ext3 filesystem One possible option with my suggested ruby-like API the_disk.all_partitions.with(type: :logical).select do |part| !part.all_encryptions.all_filesystems.with(type: :ext3).empty? end Arvin, can you please provide the bare libstorage-ng example so we save the correction step. ;-) Cheers. -- Ancor González Sosa YaST Team at SUSE Linux GmbH -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Mon, Feb 27, 2017 at 11:34:40AM +0100, Ancor Gonzalez Sosa wrote:
But let's try another less simplistic example, in which I expect at least some downcasts in the bare libstorage-ng version:
For a given disk, an array of its encrypted logical partitions containing an ext3 filesystem
One possible option with my suggested ruby-like API the_disk.all_partitions.with(type: :logical).select do |part| !part.all_encryptions.all_filesystems.with(type: :ext3).empty? end
Do you have a use-case for that function? Apart from that I am sure there are example where you need downcasts. But my point is that you should bring good examples if you complain about the Ruby bindings. ciao Arvin -- Arvin Schnell, <aschnell@suse.com> Senior Software Engineer, Research & Development SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On 02/27/2017 07:51 AM, Ancor Gonzalez Sosa wrote:
Since the original thread is already ruined with discussions about concrete API design details that were not the main topic of my question, I will try once again from scratch.
Do we need a layer between libstorage-ng and (the rest) of YaST?
I think so. Why do I?
TLDR; libstorage-ng is multilanguage (with a Ruby-agnostic API, to say the least) and YaST-agnostic. Therefore, we need a layer to close the distance between libstorage-ng and Ruby/YaST. The alternative is to have boilerplate code all along YaST (like downcast) and our own customization for the Ruby tools.
I agree. In my opinion, there are several differences between C++ and Ruby ways that justify this layer. I don't say that one is better than other, simply they are different in nature and one should not impose anything to the other one. Most probably, both APIs look similar at the end, but there are enough reasons (as was exposed) to have an intermediate layer. I think to have two different APIs is not a problem if they are the most "correct" ones (and confortable) in their respective wolds. Another positive point is that any change in libstorage API will not affect the ruby part since we have only to adapt the wrapper. -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
Dne 27.2.2017 v 08:51 Ancor Gonzalez Sosa napsal(a):
Do we need a layer between libstorage-ng and (the rest) of YaST?
I think so. Why do I?
TLDR; libstorage-ng is multilanguage (with a Ruby-agnostic API, to say the least) and YaST-agnostic. Therefore, we need a layer to close the distance between libstorage-ng and Ruby/YaST. The alternative is to have boilerplate code all along YaST (like downcast) and our own customization for the Ruby tools.
If you agree, please say so.
If you disagree, please provide counterarguments.
I do agree that this new layer between libstorage-ng and YaST Storage is the most logical way of solving most of these issues and API design problems that we have seen so far. Maybe you still remember goals for 2016 - Code Quality: ... C++ looks like C++, Ruby looks like Ruby, ... It was there for a reason ;) Thanks! Lukas -- Lukas Ocilka, Systems Management (Yast) Team Leader SLE Department, SUSE Linux Sent from my Zarma Haka 3.0 -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
participants (6)
-
Ancor Gonzalez Sosa -
Arvin Schnell -
Imobach Gonzalez Sosa -
Josef Reidinger -
José Iván López González -
Lukas Ocilka