Grub broke after upgrading Ubuntu on Azure VM from 18.04 (LTS) to 20.04 (LTS) → Recovery

Introduction

I upgraded my Azure VM Ubuntu 18.04 (LTS) to 20.04 (LTS) and rebooted, but I couldn't connect remotely with SSH indefinitely.

The Azure portal serial console is also unresponsive. The following message is displayed in the screenshot of the boot diagnosis of Azure VM in the Azure portal.

'grub_file_filters' not found

Grub seems to be broken.

environment

Upgrade procedure

I upgraded to Ubuntu 18.04 (LTS) of Azure VM by remote connection from Ubuntu of WSL by SSH and following the procedure below.

  1. Check and apply updates in advance

    $ sudo apt update
    $ sudo apt upgrade
    
  2. Perform the upgrade

    $ sudo do-release-upgrade
    
  3. Choices are basic y (YES), all upgrade suggestions accepted, LXD 4.0

  4. Reboot to complete the upgrade

  5. Unable to restart ... ('grub_file_filters' not found)

First of all, information gathering

I want to reinstall Grub and repair it, but when I was wondering what happened in the Azure VM environment, I found the following information from Microsoft.

Linux VM boots to Grub Rescue

I didn't find the corresponding information in "Recommended steps", so I followed the link.

How to recover Azure Linux virtual machines from kernel-related boot issues

I thought it might be about "Method 2: Offline repair" in "How to update the configuration file", so I looked at the link.

Troubleshoot a Linux VM by attaching the OS disk to a recovery VM with the Azure CLI

I solved the problem by creating a disk from a snapshot of the OS disk of the Azure VM in question and mounting it on the Azure VM for repair, as described in the "Recovery process overview" below. It seems that we can go by returning to the original Azure VM.

  1. Stop the affected VM.

  2. Take a snapshot from the OS disk of the VM.

  3. Create a disk from the OS disk snapshot.

  4. Attach and mount the new OS disk to another Linux VM for troubleshooting purposes.

  5. Connect to the troubleshooting VM. Edit files or run any tools to fix issues on the new OS disk.

  6. Unmount and detach the new OS disk from the troubleshooting VM. Change the OS disk for the affected VM.

Step 1: Stop the virtual machine

Step 2: Take a snapshot of the OS disk of the affected VM

Step 3: Create a disc from a snapshot

Step 4-A: Create an Azure VM to mount and repair the disk

Step 4-B: Connect the new virtual hard disk to another VM

Step 5-A: Mount the attached data disk

Step 5-B: Fix problems with new OS disks

Step 6-A: Unmount and disconnect the new OS disk

Step 6-B: Change the OS disk of the affected VM

Finally operation check

I have confirmed that I can connect to the Azure VM that I replaced with the repaired disk remotely by SSH. The upgrade to 20.04 (LTS) has also been completed.

Welcome to Ubuntu 20.04.1 LTS (GNU/Linux 5.4.0-1031-azure x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Sat Nov  7 13:14:17 JST 2020

  System load:  0.04               Processes:             139
  Usage of /:   26.2% of 28.90GB   Users logged in:       1
  Memory usage: 61%                IPv4 address for eth0: 10.0.0.5
  Swap usage:   0%

 * Introducing self-healing high availability clustering for MicroK8s!
   Super simple, hardened and opinionated Kubernetes for production.

     https://microk8s.io/high-availability

0 updates can be installed immediately.
0 of these updates are security updates.


Last login: Fri Nov  6 13:55:00 2020 from 133.200.8.0

Delete any unnecessary items such as snapshots and discs before replacement.

If you want to keep the Azure VM for repair, it's safe to leave it stopped (deallocated).

in conclusion

I'm glad I was able to repair it as a result, but I should have backed it up before the upgrade.

Recommended Posts

Grub broke after upgrading Ubuntu on Azure VM from 18.04 (LTS) to 20.04 (LTS) → Recovery
Migrate Ubuntu 18.04 LTS from HDD to smaller SSD
How to install multiple JDKs on Ubuntu 18.04 LTS
Apache2 on Ubuntu20.04 LTS
[Note] Flow from docker installation to JupyterLab startup on ubuntu
How to Install Elixir and Phoenix Framework on Ubuntu 20.04 LTS
How to install java9 on elementaryOS Freya or Ubuntu 14.04 LTS
Use virtual AP on Ubuntu 18.04 LTS to sub-monitor old iPad
Building WebGIS on Ubuntu20.04 LTS
Record video on ubuntu18.04 LTS
Installing OpenMX on Ubuntu 18.04.5 LTS
Install raspi-config on Ubuntu 20.04 (LTS)
Install WordPress 5.5 on Ubuntu 20.04 LTS