Resetting Root Password

Several methods exist to set a new root password.

A system administrator could, for example, boot the system by using a Live CD, mount the root file system from there, and edit /etc/shadow. This article explores a method that does not require the use of external media. This task is trivial if the administrator is still logged in, either as an unprivileged user but with full sudo access, or as root.

On Red Hat Enterprise Linux 9 (RHEL 9), the scripts that run from the initramfs image can be paused at certain points, to provide a root shell, and then continue when that shell exits. This script is mostly meant for debugging, and also to reset a lost root password.

Starting from RHEL 9, if you install your system from a DVD, then the default kernel asks for the root password when you try to enter maintenance mode. Thus, to reset a lost root password, you must use the rescue kernel.

To access that root shell, follow these steps:

1. Reboot the system.
2. Interrupt the boot-loader countdown by pressing any key, except Enter.
3. Move the cursor to the rescue kernel entry to boot (the entry with the rescue word in its name).
4. Press e to edit the selected entry.
5. Move the cursor to the kernel command line (the line that starts with linux).
6. Append rd.break. With that option, the system breaks just before the system hands control from the initramfs image to the actual system.
7. Press Ctrl+x to boot with the changes.
8. Press Enter to perform maintenance when prompted.

At this point, the system presents a root shell, and the root file system on the disk is mounted read-only on /sysroot.

Because troubleshooting often requires modifying the root file system, you must remount the root file system as read/write. The following step shows how the remount,rw option to the mount command remounts the file system where the new option (rw) is set.

To reset the root password, use the following procedure:

First, remount /sysroot as read/write.

sh-5.1# mount -o remount,rw /sysroot

Next, switch into a chroot jail, where /sysroot is treated as the root of the file-system tree.

sh-5.1# chroot /sysroot

In addition, set a new root password.

sh-5.1# passwd root

It is equally important to ensure that all unlabeled files, including /etc/shadow at this point, get relabeled during boot.

sh-5.1# touch /.autorelabel

Lastly, type exit twice. The first command exits the chroot jail, and the second command exits the initramfs debug shell.

Inspect Logs

Looking at the logs of previously failed boots can be useful. If the system journals persist across reboots, then you can use the journalctl tool to inspect those logs.

Remember that by default, the system journals are kept in the /run/log/journal directory, and the journals are cleared when the system reboots. To store journals in the /var/log/journal directory, which persists across reboots, set the Storage parameter to persistent in the /etc/systemd/journald.conf file.

[root@host ~]# vim /etc/systemd/journald.conf
...output omitted...
[Journal]
Storage=persistent
...output omitted...
[root@host ~]# systemctl restart systemd-journald.service

To inspect the logs of a previous boot, use the journalctl command -b option. Without any arguments, the journalctl command -b option displays only messages since the last boot. With a negative number as an argument, it displays the logs of previous boots.

[root@host ~]# journalctl -b -1 -p err

This command shows all messages that are rated as an error or worse from the previous boot.

Repair systemd Boot Issues

To troubleshoot service startup issues at boot time, Red Hat Enterprise Linux 8 and later versions provide the following tools:

Enable the Early Debug Shell

By enabling the debug-shell service with the systemctl enable debug-shell.service command, the system spawns a root shell on TTY9 (Ctrl+Alt+F9) early during the boot sequence. This shell is automatically logged in as root, so that administrators can debug the system when the operating system is still booting.

[!WARNING] Disable the debug-shell.service service when you are done debugging, because it leaves an unauthenticated root shell open to anyone with local console access.

Alternatively, to activate the debug shell during the boot by using the GRUB2 menu, proceed as follows:

1. Reboot the system.
2. Interrupt the boot-loader countdown by pressing any key, except Enter.
3. Move the cursor to the kernel entry to boot.
4. Press e to edit the selected entry.
5. Move the cursor to the kernel command line (the line that starts with linux).
6. Append systemd.debug-shell. With this parameter, the system boots into the debug shell.
7. Press ctrl+x to boot with the changes.

Use the Emergency and Rescue Targets

By appending either systemd.unit=rescue.target or systemd.unit=emergency.target to the kernel command line from the boot loader, the system enters into a rescue or emergency shell instead of starting normally. Both of these shells require the root password.

The emergency target keeps the root file system mounted read-only, while the rescue target waits for the sysinit.target unit to complete, so that more of the system is initialized, such as the logging service or the file systems.

The root user at this point cannot change /etc/fstab until the drive is remounted in a read write state with the mount -o remount,rw / command.

Administrators can use these shells to fix any issues that prevent the system from booting normally, for example, a dependency loop between services, or an incorrect entry in /etc/fstab. Exiting from these shells continues with the regular boot process.

Identify Stuck Jobs

During startup, systemd spawns various jobs. If some of these jobs cannot complete, then they block other jobs from running. To inspect the current job list, administrators can use the systemctl list-jobs command.

Any jobs that are listed as running must complete before the jobs that are listed as waiting can continue.

Conclusion

Losing root access doesn’t mean reinstalling your system. The rescue kernel with rd.break lets you reset credentials without external media—just remember to remount /sysroot read/write and create /.autorelabel for SELinux.

Combined with persistent journaling and the debug shell or emergency targets, these techniques form an essential recovery toolkit for any Linux administrator.

Keep them documented; you’ll need them when you least expect it.