Using Rescue Mode on a Server

Learn more about booting the server to Rescue.

Mounting the File System

Before performing any operations in Rescue mode, you must mount the OS file system, if this did not occur automatically. The following command is used for this:

infiltrate-root

In some cases, infiltrate-root may not work. One of the reasons may be that the RAID was not assembled.

Check how the OS file system is organized:

fdisk -l

If md devices are not specified, but sda, sdb, etc. have partitions with the Linux RAID type (in the Type field), then you need to assemble the RAID. If there are no Linux RAID partitions, it means that the OS is without RAID and you can proceed to mounting partitions according to the instructions provided below.

Assembling RAID

Sda, sdb, and so on devices with Linux RAID partitions are part of a single array. To access them, you need to assemble the array into virtual devices, which will look like /dev/md0, /dev/md1, and so on.

To do this, run the mdadm --assemble command and pass the name of the virtual device and part of the array as arguments. For example, there are 2 disks: /dev/sda and /dev/sdb. The first disk contains /dev/sda2 and /dev/sda3 partitions, and the second contains /dev/sdb2 and /dev/sdb3 partitions. All of them are of the Linux RAID type. You need to combine all partitions on different disks within the same sequence number.

An example for two disks:

mdadm --assemble /dev/md0 /dev/sda2 /dev/sdb2
mdadm --assemble /dev/md1 /dev/sda3 /dev/sdb3

There may be more disks in the RAID depending on the number of devices. An example for four disks (this is usually relevant for RAID 10):

mdadm --assemble /dev/md0 /dev/sda2 /dev/sdb2 /dev/sdc2 /dev/sdd2
mdadm --assemble /dev/md1 /dev/sda3 /dev/sdb3 /dev/sdc3 /dev/sdd3

After the assembling is complete, you can proceed to mounting the file system.

Mounting a Partition (without using LVM)

If the file system list contains only devices of the /dev/sda, /dev/sdb, and so on type, then this means that the logical volume manager (LVM) is not used. For example, if the /dev/sda diskhas a /dev/sda1 partition, then you need to run the following command to mount the partition:

mount /dev/sda1 /mnt

To mount with a root directory change, infiltrate-root is used:

infiltrate-root /dev/sda1

Mounting a Partition (using LVM)

If the file system list contains devices with root in the name (for example, /dev/mapper/vg0-root), then LVM is used, and you need to run the following command to mount the partition:

mount /dev/mapper/vg0-root /mnt

To mount with a root directory change, run:

infiltrate-root /dev/mapper/vg0-root

When using infiltrate-root, the partition will be mounted in the /newroot directory.

Mounting the File System in Windows

You can mount the file system in the same way as for Linux without using LVM by running the following command:

mount /dev/sda1 /mnt

In some cases, an error may occur:

The disk contains an unclean file system (0, 0).
Metadata kept in Windows cache, refused to mount.
Falling back to read-only mount because the NTFS partition is in an
unsafe state. Please resume and shutdown Windows fully (no hibernation
or fast restarting.)

In this case, you will not be able to write to the mounted volume. If you do not need it, then you can ignore this message. Otherwise, you need to unmount the volume and correct basic errors in the NTFS file system using the ntfsfix utility, and then repeat the mount operation:

umount /mnt
ntfsfix /dev/sda1
mount /dev/sda1 /mnt

You can also mount an NTFS partition by directly accessing the ntfs-3g component:

ntfs-3g /dev/sda1 /mnt

Resetting the Password

To reset your password in Linux:

  1. Boot the server to Rescue.
  2. Mount the OS file system.
  3. Run the utility and follow the instructions:
passwd root
  1. After entering the password, run:
sync
  1. Return to the Rescue Console by running the command:
exit

Copying Data from the Server

There are many ways to transfer data remotely, such as connecting via SFTP.

SFTP is a protocol for performing file operations over SSH. You will need an app that supports SFTP, such as FileZilla.

To access the file system on a remote server:

  1. Boot the server to Rescue.
  2. Mount the OS file system.
  3. Connect to the server using an SFTP client.

An example of connection with FileZilla:

Fill in the following fields:

  • Host — sftp://server_IP_address;
  • Username — username that can be found on the server page in the Operating system tab;
  • Password — the password from this section.

The Server files will be available in the Remote Site window of FileZilla.

Changing Server Network Settings

Changing the settings depends on the OS distribution.

Ubuntu (versions 18.04 and later)

  1. Boot the server to Rescue.

  2. Mount the OS file system.

  3. Open the netplan utility configuration file (for example, using vim):

    vi /etc/netplan/01-netcfg.yaml

    or

    vi /etc/netplan/50-cloud-init.yaml

  4. In the configuration file, change the addresses and gateway4 values to the new IP address and gateway:

    addresses: [<IP/MASK>]
gateway4: <GATEWAY>

    An example of the configuration file for the 188.93.23.36/29 subnet with server IP address 188.93.23.36 and gateway 188.93.23.33:

    Please note that the configuration file is written in YAML, which is tab- and space-sensitive. Fill it out in the same way as it was filled out earlier.

  5. Save the file and exit your text editor.

  6. In the console, enter the following:

    netplan --debug generate
netplan apply

  7. Go to the Ports tab on the server card.

  8. In the VLAN field, change the Shared value with the value of the dedicated VLAN.

CentOS

  1. Boot the server to Rescue.

  2. Mount the OS file system.

  3. To view all network adapters installed on the system, run the following:

    ip a

  4. Open the configuration file (for example, using vim):

    vi /etc/sysconfig/network-scripts/ifcfg-EN1

    Where EN1 is the name of the network adapter whose settings you want to change.

    For example, to configure the eno1 adapter, run the following:

    vi /etc/sysconfig/network-scripts/ifcfg-eno1

  5. In the configuration file, change the IPADDR and GATEWAY values to the new IP address and gateway:

    An example of the configuration file for the 188.93.23.32/29 subnet with server IP address 188.93.23.37 and gateway 188.93.23.33:

  6. Save the file and exit your text editor.

  7. If you are using CentOS 7, enter in the console:

    systemctl restart network

    In CentOS 8:

    systemctl restart NetworkManager
nmcli networking off; nmcli networking on

  8. Go to the Ports tab on the server card.

  9. In the VLAN field, change the Shared value with the value of the dedicated VLAN.

Debian / Ubuntu (versions 16.04 and earlier)

  1. Boot the server to Rescue.

  2. Mount the OS file system.

  3. Open the configuration file (for example, using vim):

    vi /etc/network/interfaces

  4. In the configuration file, change the address and gateway values to the new IP address and gateway:

    An example of the configuration file with server IP address 188.93.23.37 and gateway 188.93.23.33:

  5. Save the file and exit your text editor.

  6. In the console, enter:

    ip addr flush EN1

    Where EN1 — is the name of the network adapter whose settings were changed.

  7. Run:

    service networking restart

  8. Go to the Ports tab on the server card.

  9. In the VLAN field, change the Shared value with the value of the dedicated VLAN.

Recovering the OS Bootloader

Recovering Linux Bootloader

The main Linux bootloader is GRUB. If problems occur, you can recover the bootloader using LiveCD — boot to Rescue.

In a Live system, you need to mount all partitions in a prebuilt folder (for example, /mnt) in the same way as they were mounted on the main system.

To recover the bootloader:

  1. Start the system with LiveCD and open the terminal.

  2. Find the root partition where the system is installed using the fdisk command (in the output, find a partition with the Linux filesystem type):

fdisk -l

- If the system is installed on a partition without using LVM, then the root partition will be in `/dev/sda*`
- In the section using LVM —  `/dev/mapper`
- If the partitions are on a software RAID array - `/dev/md*` (mounted to the corresponding array)
  1. Let’s say the main system has a root / partition located in /dev/sda2 and a /boot partition located in /dev/sda1. Mount partitions in /mnt`:
mount /dev/sda2 /mnt/  
mount /dev/sda1 /mnt/boot  

If you are using a UEFI bootloader, then mount the EFI partition as well:
```
mount /dev/sda3 /mnt/boot/efi
```
Where `/dev/sda3`is the EFI partition.

_Please note that by using the `cat /mnt/etc/fstab` command, you can see the mount point in the main OS, as well as the UUID of the disk, and map it to the UUID of the disk in the Live image._
  1. Mount /sys, /proc, and /dev directories:
mount --bind /sys /mnt/sys
mount --bind /proc /mnt/proc
mount --bind /dev /mnt/dev
  1. Go to the main system environment using chroot:
chroot /mnt

If an error occurs, then use the command with /bin/bash:

chroot /mnt /bin/bash
  1. If GRUB is not installed on the disks, then install it. If installation is not required, proceed to step 7. You need to install GRUB on all disks from which you plan to boot. For example, if RAID0 is used for the boot partition:
grub-install /dev/sda
grub-install /dev/sdb

There may be more disks in the RAID depending on the number of devices.
  1. To generate a GRUB configuration file, run:
update-grub

or
```
update-grub2
```
  1. Exit the chroot environment:
exit

List of Utilities

Other utilities are also available to use Rescue mode, see the List of utilities.