[opensuse] Leap 42.1 Filesystem Problem
I get the following display on the monitor when trying to boot Leap 42.1 Welcome to emergency mode! After logging in, type "journalctl -xb to view system logs, "systemctl reboot" to reboot, "systemctl default" to try again to boot into default mode. Give root password for maintenance (or press Control-D to continue): [ 21.426046] systemd-journald[523]: File /var/log/a848146 ... e824/system.journal corrupted or uncleanly shutdown, renaming and replacing. I've issued the suggested commands without much luck in getting my system booted. Can someone help me get the file system corrected (if possible) and booted again. Thanks for any help. Terry -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
Hi Terry, On 08/07/16 20:45, Terrance Eck wrote:
I get the following display on the monitor when trying to boot Leap 42.1
Welcome to emergency mode! After logging in, type "journalctl -xb to view system logs, "systemctl reboot" to reboot, "systemctl default" to try again to boot into default mode. Give root password for maintenance (or press Control-D to continue): [ 21.426046] systemd-journald[523]: File /var/log/a848146 ... e824/system.journal corrupted or uncleanly shutdown, renaming and replacing.
I've issued the suggested commands without much luck in getting my system booted. Can someone help me get the file system corrected (if possible) and booted again.
Thanks for any help.
Terry
Which filesystem? Btrfs? Ext2/3/4? XFS? -- Bob Williams System: Linux 4.1.20-11-default Distro: openSUSE 42.1 (x86_64) Desktop: KDE Frameworks: 5.21.0, Qt: 5.5.1 and Plasma:
On 7/8/2016 3:23 PM, Bob Williams wrote:
Hi Terry,
On 08/07/16 20:45, Terrance Eck wrote:
I get the following display on the monitor when trying to boot Leap 42.1
Welcome to emergency mode! After logging in, type "journalctl -xb to view system logs, "systemctl reboot" to reboot, "systemctl default" to try again to boot into default mode. Give root password for maintenance (or press Control-D to continue): [ 21.426046] systemd-journald[523]: File /var/log/a848146 ... e824/system.journal corrupted or uncleanly shutdown, renaming and replacing.
I've issued the suggested commands without much luck in getting my system booted. Can someone help me get the file system corrected (if possible) and booted again.
Thanks for any help.
Terry Which filesystem? Btrfs? Ext2/3/4? XFS?
Btrfs. I'm hoping there is a command to fix the corrupted Btrfs file system. -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 7/8/2016 5:14 PM, Carlos E. R. wrote:
On 2016-07-08 23:47, Terrance Eck wrote:
Btrfs. I'm hoping there is a command to fix the corrupted Btrfs file system. You have not yet shown evidence that it is corrupted.
How do I show it is corrupted / uncleanly shutdown / renaming / replacing. Most likely it is an uncleanly shutdown because of a power failure while using the computer. How do I solve an uncleanly shutdown? -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Fri, Jul 8, 2016 at 5:22 PM, Terrance Eck <terry_eck@verizon.net> wrote:
On 7/8/2016 5:14 PM, Carlos E. R. wrote:
On 2016-07-08 23:47, Terrance Eck wrote:
Btrfs. I'm hoping there is a command to fix the corrupted Btrfs file system.
You have not yet shown evidence that it is corrupted.
How do I show it is corrupted / uncleanly shutdown / renaming / replacing. Most likely it is an uncleanly shutdown because of a power failure while using the computer. How do I solve an uncleanly shutdown?
Speculation is not helpful. Providing logs is what's needed. -- Chris Murphy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
08.07.2016 22:45, Terrance Eck пишет:
I get the following display on the monitor when trying to boot Leap 42.1
Welcome to emergency mode! After logging in, type "journalctl -xb to view system logs, "systemctl reboot" to reboot, "systemctl default" to try again to boot into default mode. Give root password for maintenance (or press Control-D to continue): [ 21.426046] systemd-journald[523]: File /var/log/a848146 ... e824/system.journal corrupted or uncleanly shutdown, renaming and replacing.
I've issued the suggested commands without much luck
Neither of suggested commands does anything to fix your problem. What dud you expect?
in getting my system booted. Can someone help me get the file system corrected (if possible) and booted again.
Start with showing output of "journalctl -b". -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-07-08 21:45, Terrance Eck wrote:
I get the following display on the monitor when trying to boot Leap 42.1
Welcome to emergency mode! After logging in, type "journalctl -xb to view system logs, "systemctl reboot" to reboot, "systemctl default" to try again to boot into default mode. Give root password for maintenance (or press Control-D to continue): [ 21.426046] systemd-journald[523]: File /var/log/a848146 ... e824/system.journal corrupted or uncleanly shutdown, renaming and replacing.
I've issued the suggested commands without much luck in getting my system booted.
Those commands are only for you to to decide how to find out what is wrong and decide what to do to repair the system. They don't repair anything. Translation: ^D: ignores the problem and continues - if it can. root password: get a terminal and do things on it for diagnosis and repair. ... default: boot again and hope. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
On Fri, Jul 8, 2016 at 1:45 PM, Terrance Eck <terry_eck@verizon.net> wrote:
I get the following display on the monitor when trying to boot Leap 42.1
Welcome to emergency mode! After logging in, type "journalctl -xb to view system logs, "systemctl reboot" to reboot, "systemctl default" to try again to boot into default mode. Give root password for maintenance (or press Control-D to continue): [ 21.426046] systemd-journald[523]: File /var/log/a848146 ... e824/system.journal corrupted or uncleanly shutdown, renaming and replacing.
I've issued the suggested commands without much luck in getting my system booted. Can someone help me get the file system corrected (if possible) and booted again.
Insert a USB stick and mount it to /sysroot mount /dev/sdb1 /sysroot journalctl -b > /sysroot/journal.log umount /sysroot reboot And now attach journal.log to a reply email or post it somewhere and provide the URL. -- Chris Murphy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 7/8/2016 6:35 PM, Chris Murphy wrote:
On Fri, Jul 8, 2016 at 1:45 PM, Terrance Eck <terry_eck@verizon.net> wrote:
I get the following display on the monitor when trying to boot Leap 42.1
Welcome to emergency mode! After logging in, type "journalctl -xb to view system logs, "systemctl reboot" to reboot, "systemctl default" to try again to boot into default mode. Give root password for maintenance (or press Control-D to continue): [ 21.426046] systemd-journald[523]: File /var/log/a848146 ... e824/system.journal corrupted or uncleanly shutdown, renaming and replacing.
I've issued the suggested commands without much luck in getting my system booted. Can someone help me get the file system corrected (if possible) and booted again. Insert a USB stick and mount it to /sysroot
mount /dev/sdb1 /sysroot journalctl -b > /sysroot/journal.log umount /sysroot reboot
And now attach journal.log to a reply email or post it somewhere and provide the URL.
See the attached file. It was generated by the failed system using the command journalctl -b > /mnt/journal.log I could not find where /sysroot is located so I just mounted the USB stick on /mnt. Hope the information helps someone with my filesystem problem. Terry Sorry if this post is too large for some people.
On Fri, Jul 8, 2016 at 7:06 PM, Terrance Eck <terry_eck@verizon.net> wrote:
On 7/8/2016 6:35 PM, Chris Murphy wrote:
On Fri, Jul 8, 2016 at 1:45 PM, Terrance Eck <terry_eck@verizon.net> wrote:
I get the following display on the monitor when trying to boot Leap 42.1
Welcome to emergency mode! After logging in, type "journalctl -xb to view system logs, "systemctl reboot" to reboot, "systemctl default" to try again to boot into default mode. Give root password for maintenance (or press Control-D to continue): [ 21.426046] systemd-journald[523]: File /var/log/a848146 ... e824/system.journal corrupted or uncleanly shutdown, renaming and replacing.
I've issued the suggested commands without much luck in getting my system booted. Can someone help me get the file system corrected (if possible) and booted again.
Insert a USB stick and mount it to /sysroot
mount /dev/sdb1 /sysroot journalctl -b > /sysroot/journal.log umount /sysroot reboot
And now attach journal.log to a reply email or post it somewhere and provide the URL.
See the attached file. It was generated by the failed system using the command journalctl -b > /mnt/journal.log
I could not find where /sysroot is located so I just mounted the USB stick on /mnt.
Hope the information helps someone with my filesystem problem.
Terry
Sorry if this post is too large for some people.
Jul 08 19:11:45 linux-04te kernel: BTRFS info (device sdb3): disk space caching is enabled Jul 08 19:11:45 linux-04te kernel: BTRFS: has skinny extents Jul 08 19:11:46 linux-04te systemd[1]: Mounted /sysroot. The Btrfs rootfs is fine. Jul 08 19:11:52 linux-04te systemd[1]: Starting File System Check on /dev/disk/by-uuid/df4040f2-8331-4e8b-838e-3855ebf13d5b... Jul 08 19:11:52 linux-04te systemd-fsck[830]: /sbin/fsck.xfs: XFS file system. Jul 08 19:11:52 linux-04te systemd[1]: Started File System Check on /dev/disk/by-uuid/df4040f2-8331-4e8b-838e-3855ebf13d5b. Jul 08 19:11:52 linux-04te systemd[1]: Mounting /home... Jul 08 19:11:52 linux-04te kernel: SGI XFS with ACLs, security attributes, realtime, no debug enabled Jul 08 19:11:52 linux-04te kernel: XFS (sdb4): Mounting V5 Filesystem Jul 08 19:11:53 linux-04te kernel: XFS (sdb4): Starting recovery (logdev: internal) Jul 08 19:12:03 linux-04te systemd[1]: home.mount mount process exited, code=killed status=9 Jul 08 19:12:03 linux-04te systemd[1]: Failed to mount /home. Jul 08 19:12:03 linux-04te systemd[1]: Dependency failed for Local File Systems. Jul 08 19:12:03 linux-04te systemd[1]: Dependency failed for Postfix Mail Transport Agent. Jul 08 19:12:03 linux-04te systemd[1]: Triggering OnFailure= dependencies of local-fs.target. Jul 08 19:12:03 linux-04te systemd[1]: Unit home.mount entered failed state. The XFS /home has failed to mount, journal replay failed. There is a chance you can run xfs_repair from the emergency shell if xfsprogs is in the initrd. Otherwise you'll need to boot from alternate media. If you don't have a backup of /home, you should probably do this booted from a livecd so you can use xfs_repair -n so you can see what the complaints are, and the chance of fixing it up. You're almost certainly better off using a newer xfsprogs, e.g. from a Tumbleweed LiveCD for the actual repair, but it really depends on what the problem is. As it's a v5 file system, I personally would use the most recent xfsprogs I could get my hands on. Jul 08 19:12:03 linux-04te kernel: XFS (sdb4): _xfs_buf_find: Block out of range: block 0x7fffffff8, EOFS 0x34f32800 Definitely do not use xfs_repair -l unless an XFS expert (not me) or developer says it's a good idea to do it, it's a big hammer. -- Chris Murphy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
09.07.2016 04:18, Chris Murphy пишет: ...
Jul 08 19:11:52 linux-04te systemd[1]: Starting File System Check on /dev/disk/by-uuid/df4040f2-8331-4e8b-838e-3855ebf13d5b... Jul 08 19:11:52 linux-04te systemd-fsck[830]: /sbin/fsck.xfs: XFS file system. Jul 08 19:11:52 linux-04te systemd[1]: Started File System Check on /dev/disk/by-uuid/df4040f2-8331-4e8b-838e-3855ebf13d5b. Jul 08 19:11:52 linux-04te systemd[1]: Mounting /home... Jul 08 19:11:52 linux-04te kernel: SGI XFS with ACLs, security attributes, realtime, no debug enabled Jul 08 19:11:52 linux-04te kernel: XFS (sdb4): Mounting V5 Filesystem Jul 08 19:11:53 linux-04te kernel: XFS (sdb4): Starting recovery (logdev: internal) Jul 08 19:12:03 linux-04te systemd[1]: home.mount mount process exited, code=killed status=9 Jul 08 19:12:03 linux-04te systemd[1]: Failed to mount /home. Jul 08 19:12:03 linux-04te systemd[1]: Dependency failed for Local File Systems. Jul 08 19:12:03 linux-04te systemd[1]: Dependency failed for Postfix Mail Transport Agent. Jul 08 19:12:03 linux-04te systemd[1]: Triggering OnFailure= dependencies of local-fs.target. Jul 08 19:12:03 linux-04te systemd[1]: Unit home.mount entered failed state.
The XFS /home has failed to mount, journal replay failed. There is a chance you can run xfs_repair from the emergency shell if xfsprogs is in the initrd. Otherwise you'll need to boot from alternate media. If
He already switched root at this point, so it should be available. But also note that it says "killed" which could mean it simply took too long to perform mount, although it happened too early (10 seconds after start). In any case, before attempting xfs_repair the first step would probably be to try mount it manually.
you don't have a backup of /home, you should probably do this booted from a livecd so you can use xfs_repair -n so you can see what the complaints are, and the chance of fixing it up. You're almost certainly better off using a newer xfsprogs, e.g. from a Tumbleweed LiveCD for the actual repair, but it really depends on what the problem is. As it's a v5 file system, I personally would use the most recent xfsprogs I could get my hands on.
Jul 08 19:12:03 linux-04te kernel: XFS (sdb4): _xfs_buf_find: Block out of range: block 0x7fffffff8, EOFS 0x34f32800
Definitely do not use xfs_repair -l unless an XFS expert (not me) or developer says it's a good idea to do it, it's a big hammer.
-- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Fri, Jul 8, 2016 at 10:44 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
09.07.2016 04:18, Chris Murphy пишет: ...
Jul 08 19:11:52 linux-04te systemd[1]: Starting File System Check on /dev/disk/by-uuid/df4040f2-8331-4e8b-838e-3855ebf13d5b... Jul 08 19:11:52 linux-04te systemd-fsck[830]: /sbin/fsck.xfs: XFS file system. Jul 08 19:11:52 linux-04te systemd[1]: Started File System Check on /dev/disk/by-uuid/df4040f2-8331-4e8b-838e-3855ebf13d5b. Jul 08 19:11:52 linux-04te systemd[1]: Mounting /home... Jul 08 19:11:52 linux-04te kernel: SGI XFS with ACLs, security attributes, realtime, no debug enabled Jul 08 19:11:52 linux-04te kernel: XFS (sdb4): Mounting V5 Filesystem Jul 08 19:11:53 linux-04te kernel: XFS (sdb4): Starting recovery (logdev: internal) Jul 08 19:12:03 linux-04te systemd[1]: home.mount mount process exited, code=killed status=9 Jul 08 19:12:03 linux-04te systemd[1]: Failed to mount /home. Jul 08 19:12:03 linux-04te systemd[1]: Dependency failed for Local File Systems. Jul 08 19:12:03 linux-04te systemd[1]: Dependency failed for Postfix Mail Transport Agent. Jul 08 19:12:03 linux-04te systemd[1]: Triggering OnFailure= dependencies of local-fs.target. Jul 08 19:12:03 linux-04te systemd[1]: Unit home.mount entered failed state.
The XFS /home has failed to mount, journal replay failed. There is a chance you can run xfs_repair from the emergency shell if xfsprogs is in the initrd. Otherwise you'll need to boot from alternate media. If
He already switched root at this point, so it should be available.
Good point.
But also note that it says "killed" which could mean it simply took too long to perform mount, although it happened too early (10 seconds after start). In any case, before attempting xfs_repair the first step would probably be to try mount it manually.
Yes that's true. Although xfs_repair with or without -n will catch the fact the log has not been replayed, and state that a mount attempt needs to be made before using xfs_repair which does not do log replay.
Definitely do not use xfs_repair -l unless an XFS expert (not me) or developer says it's a good idea to do it, it's a big hammer.
That should be -L. -- Chris Murphy -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 07/09/2016 01:08 PM, Chris Murphy wrote:
On Fri, Jul 8, 2016 at 10:44 PM, Andrei Borzenkov <arvidjaar@gmail.com> wrote:
09.07.2016 04:18, Chris Murphy пишет: ...
Yes that's true. Although xfs_repair with or without -n will catch the fact the log has not been replayed, and state that a mount attempt needs to be made before using xfs_repair which does not do log replay.
Definitely do not use xfs_repair -l unless an XFS expert (not me) or developer says it's a good idea to do it, it's a big hammer. That should be -L.
Well, I used the big hammer (xfs_repair -L) on /dev/sdb4 which is /home. I guess I was lucky and it seem to work. I want to thank Chris and the many members of this list for their thoughts and suggestions on how to solve the problem. Regards, Terry -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On Friday 08 July 2016 19:18:57 Chris Murphy wrote:
Jul 08 19:11:52 linux-04te systemd[1]: Starting File System Check on /dev/disk/by-uuid/df4040f2-8331-4e8b-838e-3855ebf13d5b... Jul 08 19:11:52 linux-04te systemd-fsck[830]: /sbin/fsck.xfs: XFS file system. Jul 08 19:11:52 linux-04te systemd[1]: Started File System Check on /dev/disk/by-uuid/df4040f2-8331-4e8b-838e-3855ebf13d5b. Jul 08 19:11:52 linux-04te systemd[1]: Mounting /home... Jul 08 19:11:52 linux-04te kernel: SGI XFS with ACLs, security attributes, realtime, no debug enabled Jul 08 19:11:52 linux-04te kernel: XFS (sdb4): Mounting V5 Filesystem Jul 08 19:11:53 linux-04te kernel: XFS (sdb4): Starting recovery (logdev: internal) Jul 08 19:12:03 linux-04te systemd[1]: home.mount mount process exited, code=killed status=9 Jul 08 19:12:03 linux-04te systemd[1]: Failed to mount /home. Jul 08 19:12:03 linux-04te systemd[1]: Dependency failed for Local File Systems. Jul 08 19:12:03 linux-04te systemd[1]: Dependency failed for Postfix Mail Transport Agent. Jul 08 19:12:03 linux-04te systemd[1]: Triggering OnFailure= dependencies of local-fs.target. Jul 08 19:12:03 linux-04te systemd[1]: Unit home.mount entered failed state.
The XFS /home has failed to mount, journal replay failed. There is a chance you can run xfs_repair from the emergency shell if xfsprogs is in the initrd. Otherwise you'll need to boot from alternate media. If you don't have a backup of /home, you should probably do this booted from a livecd so you can use xfs_repair -n so you can see what the complaints are, and the chance of fixing it up. You're almost certainly better off using a newer xfsprogs, e.g. from a Tumbleweed LiveCD for the actual repair, but it really depends on what the problem is. As it's a v5 file system, I personally would use the most recent xfsprogs I could get my hands on.
Jul 08 19:12:03 linux-04te kernel: XFS (sdb4): _xfs_buf_find: Block out of range: block 0x7fffffff8, EOFS 0x34f32800
Definitely do not use xfs_repair -l unless an XFS expert (not me) or developer says it's a good idea to do it, it's a big hammer.
An always valid advice is: You need a backup. If it is about a non-root partition it might be simpler and safer to just recreate the filesystem and the content from backup. Even if you manage to repair the filesystem some files might be corrupted and maybe you won't notice this until your backup also gets invalid when you overwrite the backup with your data. If you don't have a backup, this is your lesson when you learn how they are important ;-) For trying the recovery I can recommend to follow the approach what professionals in the data recovery domain and even law enforcemente would do: Copy the complete filesystem to another medium and only do recovery experiments on that image copy. For example you can call from a live medium ``` dd_rescue <path_to_your_home> <path_to_an_image_file_to_be_created_on_external_medium> ``` explicit example: ``` dd_rescue /mnt/sda/home /mnt/external_harddrive/home.img ``` All common GNU/Linux filesystem tools can also operate on image files as an argument. You can then also try to mount the partition image file like ``` mount -o bind /mnt/external_harddrive/home.img /mnt/tmp ``` Regards, Oliver -- To unsubscribe, e-mail: opensuse+unsubscribe@opensuse.org To contact the owner, e-mail: opensuse+owner@opensuse.org
On 2016-07-09 03:06, Terrance Eck wrote:
See the attached file. It was generated by the failed system using the command journalctl -b > /mnt/journal.log
I could not find where /sysroot is located so I just mounted the USB stick on /mnt.
Hope the information helps someone with my filesystem problem.
Sorry if this post is too large for some people.
Ok with me, but next time, please just upload the file to susepaste.org for a limited ammount of time. Root (/) filesystem is fine. Jul 08 19:11:51 linux-04te kernel: FAT-fs (sdb1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck. There is a fat filesystem there that needs checking. Maybe an external media? Jul 08 19:11:52 linux-04te systemd[1]: Starting File System Check on /dev/disk/by-uuid/df4040f2-8331-4e8b-838e-3855ebf13d5b... Jul 08 19:11:52 linux-04te systemd-fsck[830]: /sbin/fsck.xfs: XFS file system. Jul 08 19:11:52 linux-04te systemd[1]: Started File System Check on /dev/disk/by-uuid/df4040f2-8331-4e8b-838e-3855ebf13d5b. Jul 08 19:11:52 linux-04te systemd[1]: Mounting /home... Jul 08 19:11:52 linux-04te kernel: SGI XFS with ACLs, security attributes, realtime, no debug enabled Jul 08 19:11:52 linux-04te kernel: intel_rapl: Found RAPL domain package Jul 08 19:11:52 linux-04te kernel: intel_rapl: Found RAPL domain core Jul 08 19:11:52 linux-04te kernel: intel_rapl: Found RAPL domain dram Jul 08 19:11:52 linux-04te kernel: XFS (sdb4): Mounting V5 Filesystem ... Jul 08 19:11:53 linux-04te kernel: XFS (sdb4): Starting recovery (logdev: internal) ... Jul 08 19:12:03 linux-04te systemd[1]: home.mount mount process exited, code=killed status=9 Jul 08 19:12:03 linux-04te systemd[1]: Failed to mount /home. So it is /home which failed. If it is true that systemd kills the mount process before it finishes, I would consider that a bug. And you hit a kernel problem. A warning only: Jul 08 19:12:03 linux-04te kernel: XFS (sdb4): _xfs_buf_find: Block out of range: block 0x7fffffff8, EOFS 0x34f32800 Jul 08 19:12:03 linux-04te kernel: ------------[ cut here ]------------ Jul 08 19:12:03 linux-04te kernel: WARNING: CPU: 2 PID: 855 at ../fs/xfs/xfs_buf.c:473 _xfs_buf_find+0x2a1/0x2f0 [xfs]() I don't know the relevance of that. So, give the root's password, and try to mount home: mount -v /home Lets see the output of that, before deciding to fsck the partition. -- Cheers / Saludos, Carlos E. R. (from 13.1 x86_64 "Bottle" at Telcontar)
participants (7)
-
Andrei Borzenkov -
Bob Williams -
Carlos E. R. -
Chris Murphy -
Oliver Kurz -
Terrance Eck -
Terry Eck