[opensuse-arm] OpenStack/ARM status
Hi, I have been working towards getting OpenStack/Folsom to work on openSUSE-12.3/ARM. First, to get it to boot a VM with LXC on x86, I had to workaround https://bugzilla.novell.com/show_bug.cgi?id=814680 but when doing this on my pandaboard, it ran into another problem, claiming "internal error cannot create rule since ebtables tool is missing." even though, the ebtables tool was there, but when running libvirtd --listen in the foreground, it printed a better hint: Failure to execute command '$EBT -t nat -L' : 'FATAL: Module ebtables not found. coming from # CONFIG_BRIDGE_NF_EBTABLES is not set so I'm now going to build additional kernel modules to see how many it takes to get it working. Ciao -- Bernhard M. Wiedemann software engineer -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 04/11/2013 10:55 AM, Bernhard M. Wiedemann wrote:
Hi,
I have been working towards getting OpenStack/Folsom to work on openSUSE-12.3/ARM. First, to get it to boot a VM with LXC on x86, I had to workaround https://bugzilla.novell.com/show_bug.cgi?id=814680 but when doing this on my pandaboard, it ran into another problem, claiming "internal error cannot create rule since ebtables tool is missing." even though, the ebtables tool was there, but when running libvirtd --listen in the foreground, it printed a better hint: Failure to execute command '$EBT -t nat -L' : 'FATAL: Module ebtables not found.
coming from # CONFIG_BRIDGE_NF_EBTABLES is not set
With a recompiled kernel with ebtables-support (thanks to Dirk, it should be in the OBS ARM kernels now) I got it working! And this mail shall document how it was done: I used my PandaBoard with a 32GB SD-card (very slow, so USB-HDD is recommended) To do the setup I cloned https://github.com/SUSE-Cloud/automation and ran cloudsource=openstackfolsom bash -x automation/scripts/jenkins/qa_openstack.sh This will add the right repos and try to pull in all packages, but it errors with the cloud-compute pattern, because that one requires kvm and we don't have such a package in the 12.3 armv7l repos. So I hit Ctrl-C when I saw it, then did zypper install patterns-OpenStack-compute-node and told it to ignore kvm and install everything else anyway. Then I restarted the qa_openstack.sh call and it went through all the other downloading and setup of the OpenStack cloud on the pandaboard. In the end it imported a Debian image, which of course can not boot, because it is for i486. So I did . /etc/bash.bashrc.local to get the OpenStack environment vars needed to manage the cloud glance image-list showed the bad debian image and glance image-delete IMAGEID dropped it. LXC needs different images (a plain rootfs image) than most other methods of virtualization supported by OpenStack which need a disk image with partition table and bootloader. So I used what I had - and that was the pandaboard image. Using dd to copy out the root partition into a separate file, but I suppose you could use http://download.opensuse.org/ports/armv7hl/distribution/12.3/images/openSUSE... as well. Be sure to rmdir /selinux in the image to avoid bug 814680. I imported that image into the glance image-store with glance image-create --name=arm --is-public=True \ --property vm_mode=hvm --disk-format=raw \ --container-format=bare < rootfs.img and finally to boot this new image nova boot --image arm --flavor 1 --key_name testkey armvm you can watch it work with tail -f /var/log/nova/compute.log but as the smallest VM type still wants to reserve 512 MB this can be a bit close. Unfortunately there are no APIs to change flavor/sizes - only delete and add. The easier, but hackish way is to change it in the DB directly: su - postgres -c "psql -d nova" \d shows tables in the DB and select * from instance_types; shows the content of the table needed for this change. UPDATE instance_types SET memory_mb=256 WHERE id=1; finally halves the memory requirement. So we can delete the old VM with nova delete armvm and repeat the nova boot line from above. nova list should then show the state changing from BUILDING to Active/Running together with the VM's IP (that is not reachable from outside in this simple setup). When everything worked, ping to the VM's IP address should work and ssh root@VMIP should allow you to log in However, in my case, sshd did not start in the VM for a long time (did I mention that the SD-card was veeeery slow?), so I took an alternative entrance that is not available to all cloud users: virsh --connect lxc:/// list virsh --connect lxc:/// console instance-00000005 (press Ctrl-] to exit console) but that would not let me log in as root, before adding pts/0 to /var/lib/nova/instances/instance-00000005/rootfs/etc/securetty in the end, my VMs processes could be seen from the outside # pstree | grep -A7 lxc |-libvirt_lxc---systemd-+-cron | |-dbus-daemon | |-dhcpcd | |-login---bash | |-rsyslogd---4*[{rsyslogd}] | |-sshd | |-systemd-journal | `-systemd-logind and ssh'ing to it worked just fine as well. When you look at http://download.opensuse.org/repositories/Cloud:/OpenStack:/Folsom/openSUSE_... you might wonder how that can work without building for ARM. However, all of OpenStack is written in python, packaged as noarch and our (SUSE's) Cloud team pushed all missing dependencies into openSUSE-Factory before it became 12.3. So this is how you can have a lot of fun playing with Clouds Ciao Bernhard M. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlFnwYYACgkQSTYLOx37oWRzLACdEA0wNHNdBJt/serAKFKfP54l /KoAnixLn2DJrJhgAOvQlwvVdkpbyidl =jo1w -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
Hi Bernhard, On 12 April 2013 09:10, Bernhard M. Wiedemann <bernhardout@lsmod.de> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 04/11/2013 10:55 AM, Bernhard M. Wiedemann wrote:
Hi,
I have been working towards getting OpenStack/Folsom to work on openSUSE-12.3/ARM. First, to get it to boot a VM with LXC on x86, I had to workaround https://bugzilla.novell.com/show_bug.cgi?id=814680 but when doing this on my pandaboard, it ran into another problem, claiming "internal error cannot create rule since ebtables tool is missing." even though, the ebtables tool was there, but when running libvirtd --listen in the foreground, it printed a better hint: Failure to execute command '$EBT -t nat -L' : 'FATAL: Module ebtables not found.
coming from # CONFIG_BRIDGE_NF_EBTABLES is not set
With a recompiled kernel with ebtables-support (thanks to Dirk, it should be in the OBS ARM kernels now) I got it working!
And this mail shall document how it was done:
I used my PandaBoard with a 32GB SD-card (very slow, so USB-HDD is recommended)
To do the setup I cloned https://github.com/SUSE-Cloud/automation and ran cloudsource=openstackfolsom bash -x automation/scripts/jenkins/qa_openstack.sh
This will add the right repos and try to pull in all packages, but it errors with the cloud-compute pattern, because that one requires kvm and we don't have such a package in the 12.3 armv7l repos. So I hit Ctrl-C when I saw it, then did zypper install patterns-OpenStack-compute-node and told it to ignore kvm and install everything else anyway. Then I restarted the qa_openstack.sh call and it went through all the other downloading and setup of the OpenStack cloud on the pandaboard. In the end it imported a Debian image, which of course can not boot, because it is for i486. So I did . /etc/bash.bashrc.local to get the OpenStack environment vars needed to manage the cloud glance image-list showed the bad debian image and glance image-delete IMAGEID dropped it.
LXC needs different images (a plain rootfs image) than most other methods of virtualization supported by OpenStack which need a disk image with partition table and bootloader.
So I used what I had - and that was the pandaboard image. Using dd to copy out the root partition into a separate file, but I suppose you could use http://download.opensuse.org/ports/armv7hl/distribution/12.3/images/openSUSE... as well. Be sure to rmdir /selinux in the image to avoid bug 814680. I imported that image into the glance image-store with
glance image-create --name=arm --is-public=True \ --property vm_mode=hvm --disk-format=raw \ --container-format=bare < rootfs.img
and finally to boot this new image nova boot --image arm --flavor 1 --key_name testkey armvm
you can watch it work with tail -f /var/log/nova/compute.log
but as the smallest VM type still wants to reserve 512 MB this can be a bit close. Unfortunately there are no APIs to change flavor/sizes - only delete and add. The easier, but hackish way is to change it in the DB directly: su - postgres -c "psql -d nova" \d shows tables in the DB and select * from instance_types; shows the content of the table needed for this change. UPDATE instance_types SET memory_mb=256 WHERE id=1; finally halves the memory requirement.
So we can delete the old VM with nova delete armvm and repeat the nova boot line from above. nova list should then show the state changing from BUILDING to Active/Running together with the VM's IP (that is not reachable from outside in this simple setup). When everything worked, ping to the VM's IP address should work and ssh root@VMIP should allow you to log in However, in my case, sshd did not start in the VM for a long time (did I mention that the SD-card was veeeery slow?), so I took an alternative entrance that is not available to all cloud users: virsh --connect lxc:/// list virsh --connect lxc:/// console instance-00000005 (press Ctrl-] to exit console)
but that would not let me log in as root, before adding pts/0 to /var/lib/nova/instances/instance-00000005/rootfs/etc/securetty
in the end, my VMs processes could be seen from the outside # pstree | grep -A7 lxc |-libvirt_lxc---systemd-+-cron | |-dbus-daemon | |-dhcpcd | |-login---bash | |-rsyslogd---4*[{rsyslogd}] | |-sshd | |-systemd-journal | `-systemd-logind
and ssh'ing to it worked just fine as well.
When you look at http://download.opensuse.org/repositories/Cloud:/OpenStack:/Folsom/openSUSE_... you might wonder how that can work without building for ARM. However, all of OpenStack is written in python, packaged as noarch and our (SUSE's) Cloud team pushed all missing dependencies into openSUSE-Factory before it became 12.3.
So this is how you can have a lot of fun playing with Clouds
Ciao Bernhard M. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlFnwYYACgkQSTYLOx37oWRzLACdEA0wNHNdBJt/serAKFKfP54l /KoAnixLn2DJrJhgAOvQlwvVdkpbyidl =jo1w -----END PGP SIGNATURE----- -- To unsubscribe, e-mail: opensuse-cloud+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-cloud+owner@opensuse.org
This is excellent!! Thanks very much for all your hard work in getting this both running and documenting what's needed to do so. In an ideal world this should live on the wiki (preferably on Portal:Cloud with Portal:ARM pointing to it), or would people prefer it the other way round? Regards. Andy -- Andrew Wafaa IRC: FunkyPenguin GPG: 0x3A36312F -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
On 04/12/2013 10:34 AM, Andrew Wafaa wrote:
[...]
This is excellent!! Thanks very much for all your hard work in getting this both running and documenting what's needed to do so. In an ideal world this should live on the wiki (preferably on Portal:Cloud with Portal:ARM pointing to it), or would people prefer it the other way round?
My suggestion: Create a separate page and add the categories correctly to it: [[Category:OpenStack]] [[Category:ARM]] Thus it gets linked automatically from both portals (Portal:OpenStack in this case), Yes, this is great Andreas -- Andreas Jaeger aj@{suse.com,opensuse.org} Twitter/Identica: jaegerandi SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn,Jennifer Guild,Felix Imendörffer,HRB16746 (AG Nürnberg) GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126 -- To unsubscribe, e-mail: opensuse-arm+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse-arm+owner@opensuse.org
participants (3)
-
Andreas Jaeger -
Andrew Wafaa -
Bernhard M. Wiedemann