https://bugzilla.novell.com/show_bug.cgi?id=895204 https://bugzilla.novell.com/show_bug.cgi?id=895204#c0 Summary: processes cause kernel to hold onto chroot mount even after umount Classification: openSUSE Product: openSUSE 13.1 Version: Final Platform: Other OS/Version: Other Status: NEW Severity: Normal Priority: P5 - None Component: Kernel AssignedTo: kernel-maintainers@forge.provo.novell.com ReportedBy: aspiers@suse.com QAContact: qa-bugs@suse.de Found By: Development Blocker: --- This bug is fairly easy to reproducible by doing local builds of kiwi images using a chroot on 13.1. The first kiwi image I discovered it with was this one: https://github.com/SUSE-Cloud/suse-cloud-vagrant/tree/master/kiwi/cloud-admi... (following the instructions in the README) and then more recently with this one: https://build.suse.de/package/show/Devel:Cloud:4/suse-cloud-vagrant-admin (built via "osc build vagrantbox_images") The issue is that during kiwi's clean-up phase after building an image, even though it appears to successfully umount the bind mounts which it set up inside the chroot, actually they are not all fully umounted according to the kernel. For example: $ rm -rf /tmp/kiwi-build rm: cannot remove ‘/tmp/kiwi-build/build/image-root/proc’: Device or resource busy Even though this mountpoint no longer appears in /proc/mounts, a command such as grep -l tmp/kiwi /proc/*/task/*/mountinfo \ | sort -u \ | sed 's,^/proc/\([0-9]\+\)/task/.\+/mountinfo$,\1,' \ | xargs -r ps -fp will often reveal one or more processes still hanging onto that mount. Typically the guilty processes are one or more of the following: rtkit-daemon, colord, and the openvpn daemon. I have no clue what these have in common, but they all exhibit the capability to somehow autonomously open some kind of reference to the mount within the kernel at some point after it was mounted, and then they won't let it go until you kill them. In the case where /tmp/kiwi-build is mounted on tmpfs, this bug prevents the kernel from freeing the entire tmpfs from memory until the process hanging onto the mount is freed, as I first reported on research@ in May: http://mailman.suse.de/mlarch/SuSE/research/2014/research.2014.05/msg00005.h... The bug means that repeated kiwi builds are very problematic. But the bottom line is that the kernel is allowing a discrepancy between what /proc/mounts says, and which mounts are really being held open. And unless I misunderstood something, that shouldn't ever happen. In the case of building kiwi images via "osc build", the mountpoint will look something like this: /var/tmp/build-root/vagrantbox_images/x86_64/usr/src/packages/KIWIROOT-vmx/proc so the above "grep -l tmp/kiwi" needs to be adjusted to something like "grep -l KIWIROOT" -- Configure bugmail: https://bugzilla.novell.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are on the CC list for the bug.