[opensuse-arm] New 32-bit Arm support needed for future OBS builds
Hi, As you may know, OBS has a low number of 32-bit Arm capable workers. Some Arm servers are AArch64 only and some are still supporting 32-bit mode. The thing is new AArch64 SoC (Arm Neoverse N1) will still have the ability to run Arm 32-bit code, but only for user space (not for kernel anymore). So, we could still build for 32-bit Arm with those kind of machines, but our current setup (qemu/kvm in 32-bit mode) will not work anymore. Here are the solutions which could be used: * 64-bit qemu (with a 64-bit kernel) with armv6/armv7 rootfs * 32-bit containers on aarch64 (qemu) host * qemu without kvm: will probably be too slow I do not know how much work it would require to support one of those solutions and how many packages will be confused by a 64-bit kernel. Maybe someone have better solutions? Cheers, Guillaume -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
On Mittwoch, 5. Februar 2020, 11:16:02 CET Guillaume Gardet wrote:
Hi,
As you may know, OBS has a low number of 32-bit Arm capable workers. Some Arm servers are AArch64 only and some are still supporting 32-bit mode.
The thing is new AArch64 SoC (Arm Neoverse N1) will still have the ability to run Arm 32-bit code, but only for user space (not for kernel anymore). So, we could still build for 32-bit Arm with those kind of machines, but our current setup (qemu/kvm in 32-bit mode) will not work anymore.
Here are the solutions which could be used: * 64-bit qemu (with a 64-bit kernel) with armv6/armv7 rootfs * 32-bit containers on aarch64 (qemu) host * qemu without kvm: will probably be too slow
I do not know how much work it would require to support one of those solutions and how many packages will be confused by a 64-bit kernel.
I will object against the "without KVM" approach as OBS admin, since it means you can take over the worker. when you speak about qemu, you speak about user or system emulator? -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany (HRB 247165, AG München), Geschäftsführer: Felix Imendörffer -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
-----Original Message----- From: Adrian Schröter <adrian@suse.de> Sent: 05 February 2020 11:16 To: opensuse-arm@opensuse.org; Guillaume Gardet <Guillaume.Gardet@arm.com> Cc: Andreas Färber <afaerber@suse.de>; adrian@suse.com; nd <nd@arm.com> Subject: [opensuse-arm] Re: New 32-bit Arm support needed for future OBS builds
On Mittwoch, 5. Februar 2020, 11:16:02 CET Guillaume Gardet wrote:
Hi,
As you may know, OBS has a low number of 32-bit Arm capable workers. Some Arm servers are AArch64 only and some are still supporting 32-bit mode.
The thing is new AArch64 SoC (Arm Neoverse N1) will still have the ability to run Arm 32-bit code, but only for user space (not for kernel anymore). So, we could still build for 32-bit Arm with those kind of machines, but our current setup (qemu/kvm in 32-bit mode) will not work anymore.
Here are the solutions which could be used: * 64-bit qemu (with a 64-bit kernel) with armv6/armv7 rootfs * 32-bit containers on aarch64 (qemu) host * qemu without kvm: will probably be too slow
I do not know how much work it would require to support one of those solutions and how many packages will be confused by a 64-bit kernel.
I will object against the "without KVM" approach as OBS admin, since it means you can take over the worker.
So, against qemu without KVM (we did that in the past for armv6 on x86_64 machines, IIRC) and against containers as well?
when you speak about qemu, you speak about user or system emulator?
System emulator. Guillaume
--
Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany (HRB 247165, AG München), Geschäftsführer: Felix Imendörffer
-- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
-- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
On Mittwoch, 5. Februar 2020, 11:34:00 CET Guillaume Gardet wrote:
-----Original Message----- From: Adrian Schröter <adrian@suse.de> Sent: 05 February 2020 11:16 To: opensuse-arm@opensuse.org; Guillaume Gardet <Guillaume.Gardet@arm.com> Cc: Andreas Färber <afaerber@suse.de>; adrian@suse.com; nd <nd@arm.com> Subject: [opensuse-arm] Re: New 32-bit Arm support needed for future OBS builds
On Mittwoch, 5. Februar 2020, 11:16:02 CET Guillaume Gardet wrote:
Hi,
As you may know, OBS has a low number of 32-bit Arm capable workers. Some Arm servers are AArch64 only and some are still supporting 32-bit mode.
The thing is new AArch64 SoC (Arm Neoverse N1) will still have the ability to run Arm 32-bit code, but only for user space (not for kernel anymore). So, we could still build for 32-bit Arm with those kind of machines, but our current setup (qemu/kvm in 32-bit mode) will not work anymore.
Here are the solutions which could be used: * 64-bit qemu (with a 64-bit kernel) with armv6/armv7 rootfs * 32-bit containers on aarch64 (qemu) host * qemu without kvm: will probably be too slow
I do not know how much work it would require to support one of those solutions and how many packages will be confused by a 64-bit kernel.
I will object against the "without KVM" approach as OBS admin, since it means you can take over the worker.
So, against qemu without KVM (we did that in the past for armv6 on x86_64 machines, IIRC) and against containers as well?
armv6 was qemu inside KVM. That is a possible way. containers are not really desired security wise as well. Keep in mind that a breakage here is not only affecting the maybe-not-so-much-important armv6 distro, but really all builds in OBS.
when you speak about qemu, you speak about user or system emulator?
System emulator.
k, will not be a very fast approach, but you know that :) -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany (HRB 247165, AG München), Geschäftsführer: Felix Imendörffer -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
-----Original Message----- From: Adrian Schröter <adrian@suse.de> Sent: 05 February 2020 11:40 To: opensuse-arm@opensuse.org; Guillaume Gardet <Guillaume.Gardet@arm.com> Cc: Andreas Färber <afaerber@suse.de>; adrian@suse.com; nd <nd@arm.com> Subject: Re: [opensuse-arm] Re: New 32-bit Arm support needed for future OBS builds
On Mittwoch, 5. Februar 2020, 11:34:00 CET Guillaume Gardet wrote:
-----Original Message----- From: Adrian Schröter <adrian@suse.de> Sent: 05 February 2020 11:16 To: opensuse-arm@opensuse.org; Guillaume Gardet <Guillaume.Gardet@arm.com> Cc: Andreas Färber <afaerber@suse.de>; adrian@suse.com; nd <nd@arm.com> Subject: [opensuse-arm] Re: New 32-bit Arm support needed for future OBS builds
On Mittwoch, 5. Februar 2020, 11:16:02 CET Guillaume Gardet wrote:
Hi,
As you may know, OBS has a low number of 32-bit Arm capable workers. Some Arm servers are AArch64 only and some are still supporting 32-bit
mode.
The thing is new AArch64 SoC (Arm Neoverse N1) will still have the ability to run
Arm 32-bit code, but only for user space (not for kernel anymore).
So, we could still build for 32-bit Arm with those kind of machines, but our current setup (qemu/kvm in 32-bit mode) will not work anymore.
Here are the solutions which could be used: * 64-bit qemu (with a 64-bit kernel) with armv6/armv7 rootfs * 32-bit containers on aarch64 (qemu) host * qemu without kvm: will probably be too slow
I do not know how much work it would require to support one of those solutions and how many packages will be confused by a 64-bit kernel.
I will object against the "without KVM" approach as OBS admin, since it means you can take over the worker.
So, against qemu without KVM (we did that in the past for armv6 on x86_64 machines, IIRC) and against containers as well?
armv6 was qemu inside KVM. That is a possible way.
That is the current state. AFAIK, you cannot use KVM for armv6 qemu on x86_64. Guillaume
containers are not really desired security wise as well.
Keep in mind that a breakage here is not only affecting the maybe-not-so-much- important armv6 distro, but really all builds in OBS.
when you speak about qemu, you speak about user or system emulator?
System emulator.
k, will not be a very fast approach, but you know that :)
--
Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager
SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany (HRB 247165, AG München), Geschäftsführer: Felix Imendörffer
-- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
On Mittwoch, 5. Februar 2020, 12:26:45 CET Guillaume Gardet wrote:
-----Original Message----- From: Adrian Schröter <adrian@suse.de> Sent: 05 February 2020 11:40 To: opensuse-arm@opensuse.org; Guillaume Gardet <Guillaume.Gardet@arm.com> Cc: Andreas Färber <afaerber@suse.de>; adrian@suse.com; nd <nd@arm.com> Subject: Re: [opensuse-arm] Re: New 32-bit Arm support needed for future OBS builds
On Mittwoch, 5. Februar 2020, 11:34:00 CET Guillaume Gardet wrote:
-----Original Message----- From: Adrian Schröter <adrian@suse.de> Sent: 05 February 2020 11:16 To: opensuse-arm@opensuse.org; Guillaume Gardet <Guillaume.Gardet@arm.com> Cc: Andreas Färber <afaerber@suse.de>; adrian@suse.com; nd <nd@arm.com> Subject: [opensuse-arm] Re: New 32-bit Arm support needed for future OBS builds
On Mittwoch, 5. Februar 2020, 11:16:02 CET Guillaume Gardet wrote:
Hi,
As you may know, OBS has a low number of 32-bit Arm capable workers. Some Arm servers are AArch64 only and some are still supporting 32-bit
mode.
The thing is new AArch64 SoC (Arm Neoverse N1) will still have the ability to run
Arm 32-bit code, but only for user space (not for kernel anymore).
So, we could still build for 32-bit Arm with those kind of machines, but our current setup (qemu/kvm in 32-bit mode) will not work anymore.
Here are the solutions which could be used: * 64-bit qemu (with a 64-bit kernel) with armv6/armv7 rootfs * 32-bit containers on aarch64 (qemu) host * qemu without kvm: will probably be too slow
I do not know how much work it would require to support one of those solutions and how many packages will be confused by a 64-bit kernel.
I will object against the "without KVM" approach as OBS admin, since it means you can take over the worker.
So, against qemu without KVM (we did that in the past for armv6 on x86_64 machines, IIRC) and against containers as well?
armv6 was qemu inside KVM. That is a possible way.
That is the current state.
qemu-user-land inside of KVM, right ...
AFAIK, you cannot use KVM for armv6 qemu on x86_64.
qemu-system inside of KVM? Yes, not right now, we could teach the build script yet-another mode tough.... Question is if it really makes sense, because even slow native hardware would be faster... -- Adrian Schroeter <adrian@suse.de> Build Infrastructure Project Manager SUSE Software Solutions Germany GmbH, Maxfeldstr. 5, 90409 Nuernberg, Germany (HRB 247165, AG München), Geschäftsführer: Felix Imendörffer -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
Greetings everybody, Am 05.02.20 um 14:28 schrieb Adrian Schröter:
On Mittwoch, 5. Februar 2020, 12:26:45 CET Guillaume Gardet wrote:
-----Original Message----- From: Adrian Schröter <adrian@suse.de> Sent: 05 February 2020 11:40 To: opensuse-arm@opensuse.org; Guillaume Gardet <Guillaume.Gardet@arm.com> Cc: Andreas Färber <afaerber@suse.de>; adrian@suse.com; nd <nd@arm.com> Subject: Re: [opensuse-arm] Re: New 32-bit Arm support needed for future OBS builds
On Mittwoch, 5. Februar 2020, 11:34:00 CET Guillaume Gardet wrote:
-----Original Message----- From: Adrian Schröter <adrian@suse.de> Sent: 05 February 2020 11:16 To: opensuse-arm@opensuse.org; Guillaume Gardet <Guillaume.Gardet@arm.com> Cc: Andreas Färber <afaerber@suse.de>; adrian@suse.com; nd <nd@arm.com> Subject: [opensuse-arm] Re: New 32-bit Arm support needed for future OBS builds
On Mittwoch, 5. Februar 2020, 11:16:02 CET Guillaume Gardet wrote:
Hi,
As you may know, OBS has a low number of 32-bit Arm capable workers. Some Arm servers are AArch64 only and some are still supporting 32-bit
mode.
The thing is new AArch64 SoC (Arm Neoverse N1) will still have the ability to run
Arm 32-bit code, but only for user space (not for kernel anymore).
So, we could still build for 32-bit Arm with those kind of machines, but our current setup (qemu/kvm in 32-bit mode) will not work anymore.
Here are the solutions which could be used: * 64-bit qemu (with a 64-bit kernel) with armv6/armv7 rootfs
Is there an option missing here, namely aarch64 qemu using kvm with armv7-a kernel and userspace?
* 32-bit containers on aarch64 (qemu) host * qemu without kvm: will probably be too slow
I do not know how much work it would require to support one of those solutions and how many packages will be confused by a 64-bit kernel.
I will object against the "without KVM" approach as OBS admin, since it means you can take over the worker.
So, against qemu without KVM (we did that in the past for armv6 on x86_64 machines, IIRC) and against containers as well?
armv6 was qemu inside KVM. That is a possible way.
That is the current state.
qemu-user-land inside of KVM, right ...
AFAIK, you cannot use KVM for armv6 qemu on x86_64.
qemu-system inside of KVM?
Yes, not right now, we could teach the build script yet-another mode tough....
Question is if it really makes sense, because even slow native hardware would be faster...
-- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
-----Original Message----- From: Josua Mayer <josua.mayer97@gmail.com> Sent: 05 February 2020 16:11 To: opensuse-arm@opensuse.org Subject: Re: [opensuse-arm] Re: New 32-bit Arm support needed for future OBS builds
Greetings everybody,
Am 05.02.20 um 14:28 schrieb Adrian Schröter:
On Mittwoch, 5. Februar 2020, 12:26:45 CET Guillaume Gardet wrote:
-----Original Message----- From: Adrian Schröter <adrian@suse.de> Sent: 05 February 2020 11:40 To: opensuse-arm@opensuse.org; Guillaume Gardet <Guillaume.Gardet@arm.com> Cc: Andreas Färber <afaerber@suse.de>; adrian@suse.com; nd <nd@arm.com> Subject: Re: [opensuse-arm] Re: New 32-bit Arm support needed for future OBS builds
On Mittwoch, 5. Februar 2020, 11:34:00 CET Guillaume Gardet wrote:
-----Original Message----- From: Adrian Schröter <adrian@suse.de> Sent: 05 February 2020 11:16 To: opensuse-arm@opensuse.org; Guillaume Gardet <Guillaume.Gardet@arm.com> Cc: Andreas Färber <afaerber@suse.de>; adrian@suse.com; nd <nd@arm.com> Subject: [opensuse-arm] Re: New 32-bit Arm support needed for future OBS builds
On Mittwoch, 5. Februar 2020, 11:16:02 CET Guillaume Gardet wrote: > Hi, > > As you may know, OBS has a low number of 32-bit Arm capable workers. > Some Arm servers are AArch64 only and some are still supporting > 32-bit
mode.
> > The thing is new AArch64 SoC (Arm Neoverse N1) will still have > the ability to run Arm 32-bit code, but only for user space (not for kernel anymore). > So, we could still build for 32-bit Arm with those kind of > machines, but our current setup (qemu/kvm in 32-bit mode) will not work anymore. > > Here are the solutions which could be used: > * 64-bit qemu (with a 64-bit kernel) with armv6/armv7 rootfs
Is there an option missing here, namely aarch64 qemu using kvm with armv7-a kernel and userspace?
No, that is the problem. With newer hardware, this option will be gone as 32-bit mode will not be supported in kernel mode, so qemu/kvm cannot run armv7 kernel. Guillaume
> * 32-bit containers on aarch64 (qemu) host > * qemu without kvm: will probably be too slow > > I do not know how much work it would require to support one of > those solutions and how many packages will be confused by a 64-bit kernel.
I will object against the "without KVM" approach as OBS admin, since it means you can take over the worker.
So, against qemu without KVM (we did that in the past for armv6 on x86_64 machines, IIRC) and against containers as well?
armv6 was qemu inside KVM. That is a possible way.
That is the current state.
qemu-user-land inside of KVM, right ...
AFAIK, you cannot use KVM for armv6 qemu on x86_64.
qemu-system inside of KVM?
Yes, not right now, we could teach the build script yet-another mode tough....
Question is if it really makes sense, because even slow native hardware would be faster...
-- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
On Wed, Feb 05, 2020 at 03:15:28PM +0000, Guillaume Gardet wrote:
No, that is the problem. With newer hardware, this option will be gone as 32-bit mode will not be supported in kernel mode, so qemu/kvm cannot run armv7 kernel.
Are you sure about this? I mean, those cores will not be able to run 32-bit kernels natively, sure, but my (maybe outdated) information is that kvm always runs guest code in "user mode" and faults on privileged instructions, which are then emulated anyway. Torsten -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
-----Original Message----- From: Torsten Duwe <duwe@suse.de> Sent: 05 February 2020 16:41 To: Guillaume Gardet <Guillaume.Gardet@arm.com> Cc: Josua Mayer <josua.mayer97@gmail.com>; opensuse-arm@opensuse.org Subject: Re: [opensuse-arm] Re: New 32-bit Arm support needed for future OBS builds
On Wed, Feb 05, 2020 at 03:15:28PM +0000, Guillaume Gardet wrote:
No, that is the problem. With newer hardware, this option will be gone as 32-bit
mode will not be supported in kernel mode, so qemu/kvm cannot run armv7 kernel.
Are you sure about this? I mean, those cores will not be able to run 32-bit kernels natively, sure, but my (maybe outdated) information is that kvm always runs guest code in "user mode" and faults on privileged instructions, which are then emulated anyway.
That's what I have been told. I will double check. Cheers, Guillaume
Torsten
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
On 05/02/2020 17:22, Guillaume Gardet wrote:
-----Original Message----- From: Torsten Duwe <duwe@suse.de> Sent: 05 February 2020 16:41 To: Guillaume Gardet <Guillaume.Gardet@arm.com> Cc: Josua Mayer <josua.mayer97@gmail.com>; opensuse-arm@opensuse.org Subject: Re: [opensuse-arm] Re: New 32-bit Arm support needed for future OBS builds
On Wed, Feb 05, 2020 at 03:15:28PM +0000, Guillaume Gardet wrote:
No, that is the problem. With newer hardware, this option will be gone as 32-bit
mode will not be supported in kernel mode, so qemu/kvm cannot run armv7 kernel.
Are you sure about this? I mean, those cores will not be able to run 32-bit kernels natively, sure, but my (maybe outdated) information is that kvm always runs guest code in "user mode" and faults on privileged instructions, which are then emulated anyway.
That's what I have been told. I will double check.
Were you abe to double check? I belive we can run 32-bit VMs on 64-bit machines. The thing is that KVM for arm32 will be dropped in the future. I know we are using some arm32 HW to build packages, so we should look for an alternative. Regards, Matthias -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
19.02.2020 12:26, Matthias Brugger пишет:
On 05/02/2020 17:22, Guillaume Gardet wrote:
-----Original Message----- From: Torsten Duwe <duwe@suse.de> Sent: 05 February 2020 16:41 To: Guillaume Gardet <Guillaume.Gardet@arm.com> Cc: Josua Mayer <josua.mayer97@gmail.com>; opensuse-arm-stAJ6ESoqRxg9hUCZPvPmw@public.gmane.org Subject: Re: [opensuse-arm] Re: New 32-bit Arm support needed for future OBS builds
On Wed, Feb 05, 2020 at 03:15:28PM +0000, Guillaume Gardet wrote:
No, that is the problem. With newer hardware, this option will be gone as 32-bit
mode will not be supported in kernel mode, so qemu/kvm cannot run armv7 kernel.
Are you sure about this? I mean, those cores will not be able to run 32-bit kernels natively, sure, but my (maybe outdated) information is that kvm always runs guest code in "user mode" and faults on privileged instructions, which are then emulated anyway.
That's what I have been told. I will double check.
Were you abe to double check? I belive we can run 32-bit VMs on 64-bit machines. The thing is that KVM for arm32 will be dropped in the future. I know we are
Why it will be dropped?
using some arm32 HW to build packages, so we should look for an alternative.
Regards, Matthias
-- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
On 19/02/2020 17:21, Matwey V. Kornilov wrote:
19.02.2020 12:26, Matthias Brugger пишет:
On 05/02/2020 17:22, Guillaume Gardet wrote:
-----Original Message----- From: Torsten Duwe <duwe@suse.de> Sent: 05 February 2020 16:41 To: Guillaume Gardet <Guillaume.Gardet@arm.com> Cc: Josua Mayer <josua.mayer97@gmail.com>; opensuse-arm-stAJ6ESoqRxg9hUCZPvPmw@public.gmane.org Subject: Re: [opensuse-arm] Re: New 32-bit Arm support needed for future OBS builds
On Wed, Feb 05, 2020 at 03:15:28PM +0000, Guillaume Gardet wrote:
No, that is the problem. With newer hardware, this option will be gone as 32-bit
mode will not be supported in kernel mode, so qemu/kvm cannot run armv7 kernel.
Are you sure about this? I mean, those cores will not be able to run 32-bit kernels natively, sure, but my (maybe outdated) information is that kvm always runs guest code in "user mode" and faults on privileged instructions, which are then emulated anyway.
That's what I have been told. I will double check.
Were you abe to double check? I belive we can run 32-bit VMs on 64-bit machines. The thing is that KVM for arm32 will be dropped in the future. I know we are
Why it will be dropped?
Because it complicates the maintenance of KVM on ARM and there was a bug for a long time and no one complained, what indicates no user who is relying on a recent kernel for that. Regards, Matthias -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
participants (6)
-
Adrian Schröter -
Guillaume Gardet -
Josua Mayer -
Matthias Brugger -
Matwey V. Kornilov -
Torsten Duwe