Adding scripts and tools to SystemRescue
Custom Repair Toolkit
You can do more with SystemRescue than just repair broken systems. By adding tools and scripts, you can create a custom rescue environment that meets your needs.
Computers crash. Whether because of human error, software bugs, or hardware dying from old age, workstations and servers alike will eventually break. For this reason, every system administrator should have a digital first-aid kit to get broken systems back to working condition as soon as possible. For this task, I keep SystemRescue in my toolkit.
SystemRescue, formerly known as SystemRescueCd, is a Live Linux distribution designed to repair broken systems and handle a number of maintenance tasks. It includes tools to recover deleted files, repair broken boot managers, fix inconsistent filesystems, and more. The default set of installed programs is impressive (see Table 1 for a sample of included tools or visit the SystemRescue website [1] for the complete list). If one of your workstations or servers is rendered unbootable, you can simply boot your SystemRescue DVD or USB and have a full recovery environment available in a matter of minutes. With SystemRescue, you can repair damaged boot managers, partition tables, or filesystems. More importantly, you can extract your files from the failing computer and safely save them to a USB-attached hard drive or over a network.
Table 1
SystemRescue Tools (an Incomplete List)
Tool | Description |
---|---|
GNU ddrescue |
Copies data from devices with bad blocks (such as damaged hard drives) |
FSArchiver |
Saves and restores filesystem images |
OpenSSH server |
Controls the recovery environment from a remote machine and easily transfers files in and out of the damaged computer |
Minicom |
Connects to serial consoles |
cryptsetup |
Accesses Linux encrypted disks |
chntpw |
Resets Windows passwords |
TestDisk |
Recovers lost partitions and repairs boot sectors |
PhotoRec |
Recovers files deleted accidentally (or otherwise) |
wipe |
Deletes data securely |
Rclone |
Transfers and syncs files to a number of storage systems, including FTP and Nextcloud |
Despite SystemRescue's impressive feature set, you will eventually find yourself in a situation where the default features fall short. SystemRescue's maintainer has recognized this possibility and made it easy to create your own SystemRescue solutions to suit your needs. This article explains how to create a custom SystemRescue environment.
The Need for Customization
Two real-life scenarios call for creating a custom SystemRescue-based distribution. The most common scenario involves needing a tool that SystemRescue does not include by default. Because SystemRescue is based on Arch, it has access to Arch's repositories. If you ever need a tool, you can simply install it in your rescue environment as you would on a regular operating system. In practical terms, you can use the pacman package manager from within a running SystemRescue environment. For example, if you prefer Emacs over the default Vim, you can install it with the following commands:
# pacman -Sy # pacman -S emacs
Packages installed this way will be placed in a filesystem overlay in RAM. The packages will be available only to the rescue environment and won't touch the computer's hard drive that you are rescuing. On the downside, you will have to manually install these packages every single time you boot SystemRescue, because packages you install with pacman are not kept across reboots. Therefore, if you use an application often or expect to be performing maintenance tasks without access to Arch's repositories, you will need to include the desired packages in your custom version of SystemRescue.
Another scenario that calls for a custom SystemRescue involves a recovery environment that allows running custom scripts, performing tasks automatically, or executing services without the intervention of an administrator. For example, a server at your office breaks down and the responsible system administrator is out of town, requiring the sys admin to instruct an untrained co-worker to perform recovery via phone, which may not end well. An easier solution would be to have the sys admin fix the problem remotely by telling the co-worker to grab a recovery DVD from the shelf, put it in the server's DVD tray, reboot, and have SystemRescue offer a shell to the sys admin.
USB or DVD?
The easiest way to create a custom SystemRescue is to install it on a USB drive using a backing store [2], which is an area on the USB drive where changes to the running SystemRescue environment are written. You can install SystemRescue to a USB with a backing store, boot it, install all the packages you need, and configure all the desired scripts, and then shut it down. Your changes will be preserved across reboots this way.
While this approach is serviceable, I personally avoid using USB drives for recovery environments for a number of reasons. DVDs offer a reasonable guarantee that you will get a pristine recovery environment every time you boot, and there is value in that. (Your little brother is more likely to borrow a USB and replace your valuable data with B-movies.)
Because I want my rescue environment stored to read-only media, I can't use a backing store. Instead, I will create an ISO image containing my custom software and configurations and then burn it to a DVD.
Adding SSH Access and Persistent Keys
To illustrate how to include additional files and custom auto-started services on a SystemRescue DVD, I'll use the example of a bunch of servers in a remote office. One of the servers breaks, and the sys admin needs to make it boot SystemRescue and offer an SSH shell to connect to it from home.
While SystemRescue includes an SSH server, it is not accessible by default for two reasons: A proper password needs to be configured first, and the default firewall blocks incoming SSH connections. To remedy this, you need to create a custom SystemRescue image that does the following:
- Set a password to enable login over SSH
- Remove the firewall
- Use a persistent set of SSH fingerprints (because I want to verify the SSH server's identity when connecting to it remotely)
This custom SystemRescue image will then allow the senior sys admin to instruct a junior sys admin in the server room to boot SystemRescue, which will allow the sys admin to connect to the rescue environment without the junior admin performing any complex task.
To implement this setup, you need to download a copy of SystemRescue using a standard Linux system:
$ curl -L https://sourceforge.net/projects/systemrescuecd/files/sysresccd-x86/10.00/systemrescue-10.00-amd64.iso/download > systemrescue-10.00-amd64.iso
A helpful script by the SystemRescue creator lets you unpack the ISO image you just downloaded. You can use the script to extract the contents of the ISO file and modify them before repackaging. You can download the script as follows:
$ curl -L https://gitlab.com/systemrescue/systemrescue-sources/-/raw/main/airootfs/usr/share/sysrescue/bin/sysrescue-customize?inline=false> sysrescue-customize
Once the script, sysrescue-customize
, has been downloaded, extract the image's contents with the following command:
$ ./sysrescue-customize --unpack --source=systemrescue-10.00-amd64.iso --dest=custom
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Halcyon Creates Anti-Ransomware Protection for Linux
As more Linux systems are targeted by ransomware, Halcyon is stepping up its protection.
-
Valve and Arch Linux Announce Collaboration
Valve and Arch have come together for two projects that will have a serious impact on the Linux distribution.
-
Hacker Successfully Runs Linux on a CPU from the Early ‘70s
From the office of "Look what I can do," Dmitry Grinberg was able to get Linux running on a processor that was created in 1971.
-
OSI and LPI Form Strategic Alliance
With a goal of strengthening Linux and open source communities, this new alliance aims to nurture the growth of more highly skilled professionals.
-
Fedora 41 Beta Available with Some Interesting Additions
If you're a Fedora fan, you'll be excited to hear the beta version of the latest release is now available for testing and includes plenty of updates.
-
AlmaLinux Unveils New Hardware Certification Process
The AlmaLinux Hardware Certification Program run by the Certification Special Interest Group (SIG) aims to ensure seamless compatibility between AlmaLinux and a wide range of hardware configurations.
-
Wind River Introduces eLxr Pro Linux Solution
eLxr Pro offers an end-to-end Linux solution backed by expert commercial support.
-
Juno Tab 3 Launches with Ubuntu 24.04
Anyone looking for a full-blown Linux tablet need look no further. Juno has released the Tab 3.
-
New KDE Slimbook Plasma Available for Preorder
Powered by an AMD Ryzen CPU, the latest KDE Slimbook laptop is powerful enough for local AI tasks.
-
Rhino Linux Announces Latest "Quick Update"
If you prefer your Linux distribution to be of the rolling type, Rhino Linux delivers a beautiful and reliable experience.