The teardown crash is resolved (without any hacky work arounds). I'm not really sure what fixed it, but in the midst of some refactoring it went away. Modules can all be loaded dynamically via yast.import_module() Syntax matches ruby syntax where possible (with the major exception of symbols, since these do not exist in python). This is the biggest change from the old bindings. Client component system now allows loading of python yast modules. All ycp types are wrapped via swig and are generated, so fewer crashes, etc. Have some pydocs, but some work in progress here. I've submitted a pull request here: https://github.com/yast/yast-python-bindings/pull/5 There are some remaining issues, but overall we've made huge progress, and this code works very well. We should have feature parity now with the old bindings (feel free to point out if I've missed something), plus some new features. I'd appreciate a review and merge, since we're aiming to get these into SLE15 SP1 now. On 10/30/2017 02:37 AM, Josef Reidinger wrote:

On Fri, 20 Oct 2017 09:14:06 -0600 David Mulder wrote:

...

I've found a way to prevent the crash, although I don't completely understand why (or if this is correct).

If I initialize the UI like this prior to calling the Wizard via symbol table lookups, etc (CallYCPFunction in the old python bindings):

YUILoader::loadUI(); init_ui(YSettings::loadedUI());

Then I see no crashes or gui artifacts during run, so stack corruption seems resolved (init_ui() is same as from old python bindings).

Also, if I tear down YUIComponent when the python program completes (so, prior to the ruby code tearing it down), then I don't see the crash at shutdown, like this:

YUIComponent *c = YUIComponent::uiComponent(); if (c)     c->result(YCPVoid());

I noticed after loading the UI using YUILoader, that at shutdown I got a segfault in ui_finalizer() (within ruby bindings Yast.cc). It appears that the singleton still holds a pointer, but the pointer is invalid (so it is not null, but the pointer is now invalid). Making that call early resolves the crash. This is quite strange, as I expect uiComponent survive multiple results https://github.com/yast/yast-ycp-ui-bindings/blob/2c086e4a6631524c3c7eee1426...

what is probably problem is that yast core use many static variables which holds singletons and it is destroyed in a bit unknown order and this is probably root of issue :( Martin Vidner knows this in more details.

...

On 10/09/2017 09:22 AM, David Mulder wrote:

...

If you install my python bindings, it's easy to reproduce.

To install the bindings, first install these dependencies: yast2-ycp-ui-bindings autoconf-archive swig yast2-core-devel

Clone the repo: git clone https://github.com/dmulder/yast-python-bindings2.git

Build/install: cd yast-python-bindings2 autoreconf -if ./configure make && sudo make install

Then you'll need to add libpy2UI to your path: LD_LIBRARY_PATH=$LD_LIBRARY_PATHPATH:/usr/lib64/YaST2/plugin

Then to reproduce the failure, run `python` to open the interpreter, and run:

import yast yast.init_ui('ncurses') yast.Wizard.CreateDialog() yast.UI.CloseDialog()

It segfaults and generates a core dump.

On 10/09/2017 09:08 AM, Josef Reidinger wrote:

...

V Mon, 9 Oct 2017 08:54:35 -0600 David Mulder napsáno:

...

...

Problem is that you do not use Yast component system. In general all yast modules is attached to component system and you can call it with it. Old python bindings implement it properly with

https://github.com/yast/yast-python-bindings/blob/master/src/YCP.cc I added in use of the yast component system, for calling into the Wizard class in the ruby code (branch ruby_bind on the same project). This works, but there is now blatant stack corruption (crashing, UI artifacts, etc). Here is the crash at shutdown:

#0  0x00007f7a704ad855 in ?? () from /usr/lib64/libruby2.2.so.2.2 #1  0x00007f7a70389c4b in ?? () from /usr/lib64/libruby2.2.so.2.2 #2  0x00007f7a7044029e in ?? () from /usr/lib64/libruby2.2.so.2.2 #3  <signal handler called> #4  0x0000000000000000 in ?? () #5  0x00007f7a732e3643 in YUIComponent::result(YCPValue const&) ()    from /usr/lib64/YaST2/plugin/libpy2UI.so.2 This crash looks like somewhere in ycp-ui-bindings. It looks like null pointer access. we probably need exact location in where in result it failed. Can you add there some trivial example of such crash so I can try it myself with all debug packages?

Josef

...

#6  0x00007f7a6c58b4f0 in ?? () from /usr/lib64/ruby/vendor_ruby/2.2.0/x86_64-linux-gnu/yastx.so #7  0x00007f7a70496b4d in ?? () from /usr/lib64/libruby2.2.so.2.2 #8  0x00007f7a704a78ce in ?? () from /usr/lib64/libruby2.2.so.2.2 #9  0x00007f7a7049b3eb in ?? () from /usr/lib64/libruby2.2.so.2.2 #10 0x00007f7a704a1224 in ?? () from /usr/lib64/libruby2.2.so.2.2 #11 0x00007f7a704a5ffc in ?? () from /usr/lib64/libruby2.2.so.2.2 #12 0x00007f7a704a6323 in ?? () from /usr/lib64/libruby2.2.so.2.2 #13 0x00007f7a704a63f8 in ?? () from /usr/lib64/libruby2.2.so.2.2 #14 0x00007f7a7039573c in ?? () from /usr/lib64/libruby2.2.so.2.2 #15 0x00007f7a704a27e3 in ?? () from /usr/lib64/libruby2.2.so.2.2 #16 0x00007f7a704a3a92 in ?? () from /usr/lib64/libruby2.2.so.2.2 #17 0x00007f7a704a8bf0 in rb_eval_cmd () from /usr/lib64/libruby2.2.so.2.2 #18 0x00007f7a703a0003 in ?? () from /usr/lib64/libruby2.2.so.2.2 #19 0x00007f7a7038f55d in rb_protect () from /usr/lib64/libruby2.2.so.2.2 #20 0x00007f7a703a1376 in ?? () from /usr/lib64/libruby2.2.so.2.2 #21 0x00007f7a703ab3d4 in rb_gc_call_finalizer_at_exit () from /usr/lib64/libruby2.2.so.2.2 #22 0x00007f7a709bb80d in YRuby::~YRuby() () from /usr/lib64/YaST2/plugin/libpy2lang_ruby.so #23 0x00007f7a709bb8ee in YRuby::destroy() () from /usr/lib64/YaST2/plugin/libpy2lang_ruby.so #24 0x00007f7a709b66cd in Y2RubyComponent::~Y2RubyComponent() ()    from /usr/lib64/YaST2/plugin/libpy2lang_ruby.so #25 0x00007f7a709b67c9 in Y2RubyComponent::~Y2RubyComponent() ()    from /usr/lib64/YaST2/plugin/libpy2lang_ruby.so #26 0x00007f7a709b5aa1 in Y2CCRuby::~Y2CCRuby() () from /usr/lib64/YaST2/plugin/libpy2lang_ruby.so #27 0x00007f7a84b4e638 in __run_exit_handlers () from /lib64/libc.so.6 #28 0x00007f7a84b4e68a in exit () from /lib64/libc.so.6 #29 0x00007f7a84b37471 in __libc_start_main () from /lib64/libc.so.6 #30 0x000055950bf6c7fa in _start ()

Any thoughts?

...

which adds to python ability to call rest of component system. import is finding module like

https://github.com/yast/yast-python-bindings/blob/master/src/YCP.cc#L107

and call it with

https://github.com/yast/yast-python-bindings/blob/master/src/YCP.cc#L1238

so to call UI methods you need to import module "UI" and then with that call use method you want from UI namespace.

>> 2. yast.py is basically reimplementation of yast stuff in python. >> I am not sure if it is right way, as anything you will need you >> have to reimplement > Yes, the yast.py code is wrong. I'm trying to think of a good way > to use the ruby Wizard code, etc. You mention there is a way to be > able to call functions between yast modules? yes, as I describe above.

>> 3. your modules is not possible to call from rest of YaST. For >> ycp, perl and ruby it is possible to call it from each other. >> >> So in short. Your approach is more like wrapper for c++ parts >> ( currently only UI ) and rewrite for rest. >> >> In general if you need only UI from YaST, then I suggest to use >> direct approach and use libyui python bindings - >> https://github.com/libyui/libyui-bindings . But as I said this >> approach prevent you to use any part written in ruby and also rest >> of YaST cannot call your methods. >> >> It miss some layer from ycp-ui-bindings, but provide all needed to >> build UI. >> >> Josef >>

-- David Mulder SUSE Labs Software Engineer - Samba dmulder@suse.com SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org

Symbol issue is already resolved (I commented on the pull request). On 11/10/2017 12:42 AM, Josef Reidinger wrote:

V Thu, 9 Nov 2017 09:29:41 -0700 David Mulder napsáno:

Hi David, at the first let me say, that it is really impressive work you did.

...

The teardown crash is resolved (without any hacky work arounds). I'm not really sure what fixed it, but in the midst of some refactoring it went away. Modules can all be loaded dynamically via yast.import_module() Syntax matches ruby syntax where possible (with the major exception of symbols, since these do not exist in python). This is the biggest change from the old bindings. this is problem as I pointed out in pull request. This basically mean you cannot pass as argument symbol, so some method call cannot be constructed like (ruby variant) "UI.QueryWidget(Id(:widget1), :Value)". You probably need to introduce special construct ( or class ) for it.

...

Client component system now allows loading of python yast modules. All ycp types are wrapped via swig and are generated, so fewer crashes, etc. Have some pydocs, but some work in progress here.

I've submitted a pull request here: https://github.com/yast/yast-python-bindings/pull/5 I did review, but I am not python bindings expert, so just from ycp POV.

...

There are some remaining issues, but overall we've made huge progress, and this code works very well. We should have feature parity now with the old bindings (feel free to point out if I've missed something), plus some new features.

I'd appreciate a review and merge, since we're aiming to get these into SLE15 SP1 now. Merge is not a problem, but package is also missing in SLE15, so if you want to get it there, I suggest to do it now for GA ( as it is less bureucrazy then later in SP ) and also please do initial submit yourself in IBS, so it is clear you will be maintainer of module and not yast team. Will you please make me a maintainer for the yast-python-bindings obs and ibs projects so I can get this done right away?

Thanks Josef

-- David Mulder SUSE Labs Software Engineer - Samba dmulder@suse.com SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: yast-devel+unsubscribe@opensuse.org To contact the owner, e-mail: yast-devel+owner@opensuse.org