Am 05.05.2015 um 10:59 schrieb Andreas Schwab:

Dirk Müller writes:

...

it seems recently qemu-linux-user 2.3.0 has been accepted to factory, which however immediately crashes on startup,

Why wasn't it tested?

Good question. Why does this suddenly depend on Factory? I intentionally canceled two auto-submissions so that there were ~4 weeks of testing qemu in Virtualization. According to my information, openSUSE:Factory:ARM uses qemu-linux-user from Virtualization, which means that if there were a breakage we should've heard that much earlier. If that is silently changed behind my back, blame yourselves. Andreas -- SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Jennifer Guild, Dilip Upmanyu, Graham Norton; HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org

Am 06.05.2015 um 12:22 schrieb Dirk Müller:

Hi Andreas,

...

...

Why wasn't it tested? Good question. Why does this suddenly depend on Factory?

IIRC this was changed March 23rd based on your announcement that there are issues with 2.3.0-rc0.

Then that was a clear miscommunication, as I immediately committed the fix to Virtualization once I got the patch. I only asked to check whether retriggering any failed builds fixes them. Regards, Andreas -- SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Jennifer Guild, Dilip Upmanyu, Graham Norton; HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org

On 06.05.15 14:30, Ludwig Nussel wrote:

Dirk Müller wrote:

...

...

...

Why wasn't it tested? Good question. Why does this suddenly depend on Factory?

IIRC this was changed March 23rd based on your announcement that there are issues with 2.3.0-rc0.

...

openSUSE:Factory:ARM uses qemu-linux-user from Virtualization, which means that if there were a breakage we should've heard that much earlier. If that is silently changed behind my back, blame yourselves.

Its unfortunately not immediately noticeable that qemu-linux-user is broken, since packages never end up in the "failed" state (this was a thinko from our discussion yesterday), since the build service thinks its a bad worker and restarts the job on a different host, only to fail there the same way. It is only noticed then by an admin seeing that the cluster is flooded with jobs that never succeed and other things get behind.

so I do recommend to do an automated testing outside the buildservice instead, which gives you notifications and enough time to fix so that the stuff that enters Factory is working.

Would it be possible to cross compile a small arm binary on x86 and run it through qemu-linux-user? Something like

$ zypper in gcc-whatever qemu-something $ gcc somecode.c $ qemu-something a.out; echo $?

That should be easy to run in openQA...

Something that simple could even be part of %check in the package ;). Unfortunately cross compilers aren't really in openSUSE yet IIUC. Alex -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org

Ludwig Nussel writes:

Would it be possible to cross compile a small arm binary on x86 and run it through qemu-linux-user?

All that would be needed is to enable Virtualization for openSUSE:Factory:ARM/armv6l. Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org

Am 06.05.2015 um 14:59 schrieb Andreas Schwab:

Dirk Müller writes:

...

Its unfortunately not immediately noticeable that qemu-linux-user is broken, since packages never end up in the "failed" state (this was a thinko from our discussion yesterday), since the build service thinks its a bad worker and restarts the job on a different host, only to fail there the same way. It is only noticed then by an admin seeing

You don't need to be admin to notice the problem.

I just verified, and on my local system using qemu-linux-user from Virtualization I am able to run qemu-x86_64 /bin/ls just fine. So it does not seem to be a general qemu-linux-user problem. Andreas -- SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Jennifer Guild, Dilip Upmanyu, Graham Norton; HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org

Am 06.05.2015 um 15:05 schrieb Andreas Schwab:

Andreas Färber writes:

...

Virtualization I am able to run qemu-x86_64 /bin/ls just fine.

This is about qemu-arm.

You claimed yesterday that it has not been tested at all; I am thus telling you that it does pass a smoke test on my system. Dirk did not bother to CC me originally, and none of you have so far pointed to any particular breakage. According to my OBS mails, there were two gnuradio failures over the weekend plus two pihwm failures. None of them show qemu-arm misbehaving! One is about undefined references and the other is about some libtool version mismatch. So I haven't the foggiest why you are so upset! Last time a qemu-linux-user breakage came up, I asked for people to contribute tests to avoid things breaking silently in the future - no one bothered, so don't you dare cry about lack of testing now. I have tried adding a trivial test: %check %{qemu_arch}-linux-user/qemu-%{qemu_arch} %_bindir/ls > /dev/null but this always fails for all architectures and repos in OBS - sometimes with segfaults, sometimes without - whereas it succeeds with exit code 0 on my local 13.2 x86_64 system. So I am clueless what's going on here - some difference between OBS and my system apparently. https://build.opensuse.org/package/show/home:a_faerber:branches:Virtualizati... Andreas -- SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Jane Smithard, Jennifer Guild, Dilip Upmanyu, Graham Norton; HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org

On 06.05.15 23:02, Dirk Müller wrote:

Hi,

...

Dirk did not bother to CC me originally, and none of you have so far pointed to any particular breakage.

There is no "original" mail other than this thread, really, so you're not left out. I didn't bother to debug it further since it was broken everywhere on every occassion including on /bin/false. Here's the backtrace:

Program received signal SIGSEGV, Segmentation fault.

thunk_convert (dst=dst@entry=0x7fffffffcbe0, src=0x4000a1f170,

type_ptr=0x6051b20c , type_ptr@entry=0x6051b204 ,

to_host=to_host@entry=1) at /home/abuild/rpmbuild/BUILD/qemu-2.3.0/thunk.c:273

273 (*se->convert[to_host])(dst, src);

(gdb) bt

#0 thunk_convert (dst=dst@entry=0x7fffffffcbe0, src=0x4000a1f170,

type_ptr=0x6051b20c , type_ptr@entry=0x6051b204 ,

to_host=to_host@entry=1) at /home/abuild/rpmbuild/BUILD/qemu-2.3.0/thunk.c:273

#1 0x0000000060038358 in do_ioctl (arg=274888520048, cmd=<optimized out>, fd=<optimized out>)

at /home/abuild/rpmbuild/BUILD/qemu-2.3.0/linux-user/syscall.c:3940

#2 do_syscall (cpu_env=cpu_env@entry=0x625a5bd0, num=16, arg1=<optimized out>,

arg2=<optimized out>, arg3=274888520048, arg4=<optimized out>, arg5=274901073728,

arg6=274888522607, arg7=0, arg8=0)

at /home/abuild/rpmbuild/BUILD/qemu-2.3.0/linux-user/syscall.c:6281

#3 0x00000000600298b6 in cpu_loop (env=env@entry=0x625a5bd0)

at /home/abuild/rpmbuild/BUILD/qemu-2.3.0/linux-user/main.c:305

#4 0x0000000060003676 in main (argc=<optimized out>, argv=<optimized out>,

envp=<optimized out>) at /home/abuild/rpmbuild/BUILD/qemu-2.3.0/linux-user/main.c:4419

(gdb) p se->convert[to_host]

$1 = (void (*)(void *, const void *)) 0xbabababababababa

which means the pointer has been free'ed already. Overall this points out that

0037-linux-user-Allocate-thunk-size-dyna.patch is the culprit.

Bleks. I'm still waiting for the day when I write a patch and it just works. The thunk framework implicitly assumed that the se->convert fields are initialized to 0. This did happen before my patch when the thunk cache resided in the bss section. Now we're allocating it dynamically and so it may get filled with garbage (correctly tested by the malloccheck thing). The easy fix is to s/g_new/g_new0/ to expose the same allocation semantics as before. I've changed the code accordingly and submitted a fixed qemu package to the Virtualization project. Alex -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org

HI Andreas,

...

its a bad worker and restarts the job on a different host, only to fail there the same way. It is only noticed then by an admin seeing You don't need to be admin to notice the problem.

well, either that or actively waiting for a job to start, reloading the webui often enough so that you can catch the build log as it happens before it gets reverted.. I wouldn't say many people are able to do that :-) Greetings, Dirk -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org