[yast-devel] RFC - New Storage API
Hi, after the announcement of my ideas for a Storage Redesign in YaST I present now the first draft of the API that libstorage will provide. Remember that this API should be used by all YaST modules using functions from yast2-storage so far since the infamous target map will vanish. Device Graph ------------ The basic data structure is a directed graph of device objects as already explained (https://github.com/aschnell/libstorage-bgl-eval/wiki). Compared to the target map there are a few new objects, e.g. PartitionTable, Encryption and Filesystem. How to get a Device Graph ------------------------- Libstorage works with several device graphs, mainly the "probed" and the "current". Somehow you will have access to the main storage objects and can use that to get a device graph: DeviceGraph* device_graph = storage->getCurrent(); Manipulating the Device Graph ----------------------------- There are "constructors" for all concrete objects (e.g. Ext4 and Swap are concrete while Filesystem is abstract). Disk* sda = Disk::create(device_graph, "/dev/sda"); The device_graph is the owner of the objects. Creating disks will of course only be allowed in the image/KIWI mode. Most objects will be created by calling member functions of the parent: PartitionTable* gpt = sda->createPartitionTable(GPT); Partition* sda1 = gpt->createPartition("/dev/sda1"); // parameter will change Manipulating is done with member functions, e.g.: filesystem->addMountPoint("/boot"); There is also a low level interface where the vertices and edges of the graph are manipulated directly, e.g.: PartitionTable* gpt = Gpt::create(device_graph); // creates vertex Using::create(device_graph, sda, gpt); // creates edge Partition* sda1 = Partition::create("/dev/sda1"); // creates vertex Subdevice::create(device_graph, gpt, sda1); // creates edge But this low level interface should be reserved for the expert partitioner if needed at all. Querying the Device Graph ------------------------- Generic functions to query relatives of a device: vector<Device*> children = device->getChildren(); Here getChildren, getParents, getSiblings, getDescendants, getAncestors, getLeafs and getRoots is available. Special functions to query relatives of a device, e.g. the partitions of a partition table: vector<Partition*> partitions = gpt->getPartitions(); Functions to find specific devices, e.g. filesystems by mount point: Filesystem::findByMountPoint(device_graph, "/boot"); Combining query functions in a complex example: Suppose the bootloader has to know whether the device mounted at "/" uses a LVM logical volume: for (const Filesystem* filesystem : Filesystem::findByMountPoint(device_graph, "/")) { for (const Device* device : filesystem->getAncestors(false)) { if (dynamic_cast<const LvmLv*>(device)) cout << "mount point \"/\" somehow uses a logical volume" << endl; } } https://github.com/aschnell/libstorage-bgl-eval/blob/master/examples/find1.c... Bindings -------- I have swig generated bindings for Ruby, Python and Perl5. The repository contains a few examples for each of them: https://github.com/aschnell/libstorage-bgl-eval/tree/master/bindings Help ---- The Makefile.am for Perl bindings needs improvement and the bindings should get a tiny testsuite. RFC --- If you see problems with this API for your module, generally miss something or have ideas for improvement please comment now. ciao Arvin -- Arvin Schnell, <aschnell@suse.de> Senior Software Engineer, Research & Development SUSE LINUX GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Wed, Dec 03, 2014 at 11:07:34AM +0100, Arvin Schnell wrote:
Bindings --------
I have swig generated bindings for Ruby, Python and Perl5. The repository contains a few examples for each of them:
https://github.com/aschnell/libstorage-bgl-eval/tree/master/bindings
Hi Arvin, I tried to review the Ruby bindings with the goal of making them readable to a Rubyist. The method naming convention, in camelCase, looks out of place in Ruby. It should be possible to autoconvert it to snake_case via swig -ruby -autorename example.i http://www.swig.org/Doc3.0/Ruby.html#Ruby_nn27 - sda1 = gpt.createPartition("/dev/sda1") + sda1 = gpt.create_partition("/dev/sda1") Similarly, though less automatable, it would be good to rename getters and setters: http://www.swig.org/Doc3.0/Ruby.html#Ruby_nn31 - partition.getNumber() + partition.number Abbreviations can be useful to save typing and screen space but IMHO they should be avoided in class names: - Storage::BlkDevice::find(...) + Storage::BlackDevice.find(...) (A week ago I and Balazs were trying to debug string handling in Ruby VM. It took me several minutes to realize that what I thought was "capability" meant "capacity", and "terminal" was in fact "terminator". They were declared as "capa" and "term" without an explanatory comment.) Small improvements in style in the examples: - Storage::Disk::create(device_graph, "/dev/sda") + Storage::Disk.create(device_graph, "/dev/sda") - print "foo\n" + puts "foo" Please avoid booleans in the API, they are hard to understand even if you have learned the domain and the API before. I haven't seen a single mention od Ruby symbols in the Swig docs, I guess custom typemaps would be needed for that :-( - sda.getDescendants(false) + sda.descendants(:direct) # or Storage::DESCENDANTS_DIRECT For polymorphism, we have these 6 casts in Storage::Device: (which return nil if the instance is not of that type) #castToDisk #castToPartitionTable #castToPartition #castToLvmVg #castToLvmLv #castToFilesystem While the ideal OO design is to avoid casts and use method polymorphism, I don't know how doable it is in this case. A more rubyish method names would be #to_disk #to_partition_table #to_lvm_vg -- Martin Vidner, Cloud & Systems Management Team http://en.opensuse.org/User:Mvidner Kuracke oddeleni v restauraci je jako fekalni oddeleni v bazenu
On Fri, Dec 05, 2014 at 02:06:26PM +0100, Martin Vidner wrote:
On Wed, Dec 03, 2014 at 11:07:34AM +0100, Arvin Schnell wrote:
Bindings --------
I have swig generated bindings for Ruby, Python and Perl5. The repository contains a few examples for each of them:
https://github.com/aschnell/libstorage-bgl-eval/tree/master/bindings
Hi Martin, thanks for your review.
The method naming convention, in camelCase, looks out of place in Ruby. It should be possible to autoconvert it to snake_case via swig -ruby -autorename example.i http://www.swig.org/Doc3.0/Ruby.html#Ruby_nn27
- sda1 = gpt.createPartition("/dev/sda1") + sda1 = gpt.create_partition("/dev/sda1")
I have enabled autorename for Ruby.
Similarly, though less automatable, it would be good to rename getters and setters: http://www.swig.org/Doc3.0/Ruby.html#Ruby_nn31
- partition.getNumber() + partition.number
Well, that's indeed a lot of work. Having to take care of three bindings I think it's too much. Also I want to keep the SWIG interface file identical for all three bindings if possible.
Abbreviations can be useful to save typing and screen space but IMHO they should be avoided in class names:
- Storage::BlkDevice::find(...) + Storage::BlackDevice.find(...)
In general I agree but "blk" is very common in the storage are, e.g. S_ISBLK and BLKGETSIZE in C headers or lsblk and blkid on the command line.
Small improvements in style in the examples:
- Storage::Disk::create(device_graph, "/dev/sda") + Storage::Disk.create(device_graph, "/dev/sda")
- print "foo\n" + puts "foo"
OK.
Please avoid booleans in the API, they are hard to understand even if you have learned the domain and the API before. I haven't seen a single mention od Ruby symbols in the Swig docs, I guess custom typemaps would be needed for that :-(
- sda.getDescendants(false) + sda.descendants(:direct) # or Storage::DESCENDANTS_DIRECT
Looks difficult to achieve.
For polymorphism, we have these 6 casts in Storage::Device: (which return nil if the instance is not of that type) #castToDisk #castToPartitionTable #castToPartition #castToLvmVg #castToLvmLv #castToFilesystem While the ideal OO design is to avoid casts and use method polymorphism, I don't know how doable it is in this case.
Downcasting should not be needed often since querying e.g. the partitions of a partition table of course returns pointers to partitions not devices. The casts are only needed when you do a generic query on the device graph and then want to know more about the returned devices. Just a note: Using the class name instead of downcasting is wrong. E.g. there could be three distinct classes for partitions on MSDOS, GPT and DASD instead of just one. Code using if device.get_class_name == "Partition" will then fail if device is actually "GptPartition" while if to_partition(device) will still work. I will add this note to the final documentation since I want the freedom to change such things later on.
A more rubyish method names would be #to_disk #to_partition_table #to_lvm_vg
I have removed the "cast" from the names to together with autorename it's what you want now. Regards, Arvin -- Arvin Schnell, <aschnell@suse.de> Senior Software Engineer, Research & Development SUSE LINUX GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Fri, Dec 05, 2014 at 03:37:02PM +0100, Arvin Schnell wrote:
On Fri, Dec 05, 2014 at 02:06:26PM +0100, Martin Vidner wrote:
On Wed, Dec 03, 2014 at 11:07:34AM +0100, Arvin Schnell wrote:
Bindings --------
I have swig generated bindings for Ruby, Python and Perl5. The repository contains a few examples for each of them:
Similarly, though less automatable, it would be good to rename getters and setters: http://www.swig.org/Doc3.0/Ruby.html#Ruby_nn31
- partition.getNumber() + partition.number
Well, that's indeed a lot of work. Having to take care of three bindings I think it's too much. Also I want to keep the SWIG interface file identical for all three bindings if possible.
Maybe it's not that much work after all since rename supports regexs, an example from the documentation: %rename("%(regex:/^(Set|Get)(.*)/\\2/)s") ""; But then the autorename does not work anymore and I couldn't figure out how to get it working. I asked Klaus about this. He takes a different approach with Swig, starting with an empty interface description and then adding what is needed specially tailored for the target language. The result is nicer bindings at the cost of some extra work. I'm considering using that approach but I will need someone for every target language tailoring the bindings and extending the (now existing) testsuites. Regards, Arvin -- Arvin Schnell, <aschnell@suse.de> Senior Software Engineer, Research & Development SUSE LINUX GmbH, GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer, HRB 21284 (AG Nürnberg) Maxfeldstraße 5 90409 Nürnberg Germany -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
On Mon, 8 Dec 2014 11:37:16 +0100 Arvin Schnell <aschnell@suse.de> wrote:
On Fri, Dec 05, 2014 at 03:37:02PM +0100, Arvin Schnell wrote:
On Fri, Dec 05, 2014 at 02:06:26PM +0100, Martin Vidner wrote:
On Wed, Dec 03, 2014 at 11:07:34AM +0100, Arvin Schnell wrote:
Bindings --------
I have swig generated bindings for Ruby, Python and Perl5. The repository contains a few examples for each of them:
Similarly, though less automatable, it would be good to rename getters and setters: http://www.swig.org/Doc3.0/Ruby.html#Ruby_nn31
- partition.getNumber() + partition.number
Well, that's indeed a lot of work. Having to take care of three bindings I think it's too much. Also I want to keep the SWIG interface file identical for all three bindings if possible.
Maybe it's not that much work after all since rename supports regexs, an example from the documentation:
%rename("%(regex:/^(Set|Get)(.*)/\\2/)s") "";
But then the autorename does not work anymore and I couldn't figure out how to get it working.
I asked Klaus about this. He takes a different approach with Swig, starting with an empty interface description and then adding what is needed specially tailored for the target language. The result is nicer bindings at the cost of some extra work.
I'm considering using that approach but I will need someone for every target language tailoring the bindings and extending the (now existing) testsuites.
Regards, Arvin
If you use Klaus approach, then I think it is better to not use at all SWIG and write directly ruby bindings. For C++ it can help to use rice[1][2]. Josef [1] https://github.com/jasonroelofs/rice [2] http://www.ibm.com/developerworks/library/os-extendruby/ -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org
participants (3)
-
Arvin Schnell -
Josef Reidinger -
Martin Vidner