Ok, inevitably, it will come to pass that if you are testing out Alpha Releases, you will experience at least one time in your life where the system will not boot.
Recently, while working on Karmic (Ubuntu 9.10), I did an update an rebooted into a non functional system. Apparently just before freeze for Alpha 6, there were some uploads which caused the build systems to not completely build all the necessary packages. Hence I had a partial update.
Anyway, things seem to be resolved now, so on to recovery. I am documenting this mostly so I remember how to do it again (copy/paste is your friend) and maybe it will be useful to others.
First, you need a live CD/DVD/USB running your favorite Linux. In my case, it's Kubuntu. I used my usb-creator-kde to make a live USB from a known working and recent release.
After booting into your live environment, you need to open a terminal and switch to mount your partitions under /mnt.
To determine the partitions, you can use fdisk as follows:
ubuntu@ubuntu:~$ sudo fdisk -l
Disk /dev/sda: 320.1 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xd88dfd16
Device Boot Start End Blocks Id System
/dev/sda1 * 1 61 489951 83 Linux
/dev/sda2 62 38913 312078690 5 Extended
/dev/sda5 62 1306 10000431 82 Linux swap / Solaris
/dev/sda6 1307 7385 48829536 83 Linux
/dev/sda7 7386 38913 253248628+ 83 Linux
Disk /dev/sdb: 2057 MB, 2057306112 bytes
64 heads, 62 sectors/track, 1012 cylinders
Units = cylinders of 3968 * 512 = 2031616 bytes
Disk identifier: 0x0000a283
Device Boot Start End Blocks Id System
/dev/sdb1 * 1 1012 2007777 b W95 FAT32
sdb is my USB drive and sda is my HDD. sda1 is /boot, sda6 is / and sda7 is home. Yours will likely be different.
After you figure out your partitions, you will mount them as follows:
ubuntu@ubuntu:~$ sudo mount /dev/sda6 /mnt
ubuntu@ubuntu:~$ sudo mount /dev/sda1 /mnt/boot
ubuntu@ubuntu:~$ sudo mount /dev/sda7 /mnt/home
Verify everything looks correct before proceeding (i.e. you got the partitions correct and mounted in the correct places).
Next, you need to take care of /proc and /dev as follows:
ubuntu@ubuntu:~$ sudo mount -t proc none /mnt/proc
ubuntu@ubuntu:~$ sudo mount -o bind /dev /mnt/dev
Ok, now you have setup your chroot environment, and may change into it:
ubuntu@ubuntu:~$ sudo chroot /mnt
At this point, you should be able to update your system, add/remove packages using apt-get, or edit any files you need to fix the system.
In my case, a simple apt-get update && apt-get upgrade will fix things.
After making your changes you need to exit the chroot and unmount the partitions in the reverse order.
Now you should be able to safely exit the live system and reboot.
Hope this helps.
Hi, Karmic is Ubuntu 9.10 - instead of Ubuntu 9.04
ReplyDeleteOnce? Ha, you underestimate the ineptitude of an average user like myself, sir, and I resemble that remark (to rip off an old line from Looney Toons). I've already killed one of my systems twice dual booting alpha releases, and there have only been five so far! But it hasn't stopped me from doing it anyway, good trooper that I am. Anyway, I hate having to track down instructions after the fact, so thanks for the useful info. I've bookmarked this on all my machines for the next time it happens (and it will happen, if I know me). It'll be that much easier to just pull up your blog post. Thanks again!
ReplyDeleteHey Jullian, yeah, I knew that. That's what I get for flash posting without "reviewing" first :)
ReplyDeleteHi, can you explain what you mean about "setting up your chroot environment". I don't understand this concept or why it's needed. Thanks
ReplyDeleteIf you have any free space left, you could also set up a minimal rescue system on hdd. Unless you waste your /boot and or mbr this is imo a better choice.
ReplyDeleteAnonymous #1 :)
ReplyDeleteSetting up the chroot environment are the steps outlined in the post. I.e. mounting the HDD partitions and /proc and /dev to a temportary location so you can run the chroot (change root) command. This makes it so that you can work in the environment of the failed system via the live booted system.
Anonymous #2:
Good point for those who want preventative measures. It would be nice to have a procedure to set this up ahead of time. Feel free to post how.
However, for those who do not have such a system, my process is all you need to recover.
edit grub to boot rw and init=/bin/bash
ReplyDeletethis will get you to a command line, then you can get network (if you are plugged in) by doing 'dhclient &'. Then you and apt-get update and upgrade :)
Then, to get past the stuff if you are using encrypted drives with the latest mess in karmic, add a file to /etc/init/ called shell.conf that contains the following lines:
start on startup
console owner
exec /bin/bash
This will get you a command line at boot, only for those with encrypted drives of course, then you can do the following to get up and running:
/etc/init.d/cryptdisks start
Enjoy! :)
@nixternal:
ReplyDeleteWhen your system really doesn't want to boot (like with yesterday's broken updates with karmic), your method doesn't work (my system hung after loading scripts and wouldn't go further). Chrooting from livecd/liveusb was the only solution that worked.
Nixternal, thanks very much for your help regarding encrypted LVM.
ReplyDeleteI'm having a little trouble understanding what you mean about the GRUB bits:
"edit grub to boot rw and init=/bin/bash"
Where exactly do we put that? I've tried a few things and none seemed to work for me -- I still can not access my encrypted drive.
Can you be a little more verbose on how the process is done? I apologize for my ignorance on GRUB.
Many thanks,
Mike
Thx for your artikel. It helps me.
ReplyDeletefor the question
"edit grub to boot rw and init=/bin/bash":
If you choose the booting kernel in the grub menu, then don't press enter and press 'e' for edit instead. After that add at the end of "kernel = ...." this "init=/bin/bash"
cu