diff mbox series

[isar-cip-core,1/2] README.control-system-backup.md: Add steps to explain control system backup

Message ID 20220909080731.10848-2-Sai.Sathujoda@toshiba-tsip.com (mailing list archive)
State Handled Elsewhere
Headers show
Series Patches to meet IEC CR 7.3 | expand

Commit Message

Sai.Sathujoda@toshiba-tsip.com Sept. 9, 2022, 8:07 a.m. UTC
From: Sai <Sai.Sathujoda@toshiba-tsip.com>

Control system backup is an IEC security requirement to backup critical
systems for disaster recovery and system migration program.

This document demonstrates how control system backup can be taken using
'duplicity', and also provide steps to check the integrity and restore
the backups.

Signed-off-by: Sai <Sai.Sathujoda@toshiba-tsip.com>
---
 doc/README.control-system-backup.md | 194 ++++++++++++++++++++++++++++
 1 file changed, 194 insertions(+)
 create mode 100644 doc/README.control-system-backup.md
diff mbox series

Patch

diff --git a/doc/README.control-system-backup.md b/doc/README.control-system-backup.md
new file mode 100644
index 0000000..4f4f9e1
--- /dev/null
+++ b/doc/README.control-system-backup.md
@@ -0,0 +1,194 @@ 
+# Overview
+This document explains how to take system-level backup, verify integrity, and restore the backup file. It also mentions the demonstration which has to be carried out using **duplicity** to perform both local and remote backups.
+
+# Description
+There is always a chance for the data to get corrupted or lost due to some unknown reasons. In those situations, system level backup is helpful so that if any data is lost then it can be restored back. 
+If there is a lot of space available in the system then **Local backup** can be considered. But in some cases space will not be enough if we opt for daily backup. In that situation, it is better to choose remote backup where all the data in the system will be stored on a remote server.
+
+**Duplicity** provides facilities like system-level backup, encryption of the data taken as a backup, verification of data integrity before restore and finally the restoration of the data from the backup. 
+
+# Pre-Requisites
+1. CIP security image which includes **duplicity** package, to build the image follow the steps described in the [README.security-testing.md](./README.security-testing.md#build-cip-security-linux-image).
+2. Additional storage in case of local backup, (Add the below lines in **/kas/opt/security.yml** file to increase the rootfs size).
+     ```
+     local_conf_header:
+          security_image_size: |
+          ROOTFS_EXTRA = "8192"
+     ```
+3. Remote machine with Linux OS to store remote backups.
+
+# Local Backup & Restore
+For the first backup taken it will be a **full backup** but after that it will be incremental backups. If full backups are to be taken explicitly then add "full" in the command.
+
+The user can choose the data needed to be taken as backup. For example, in the below command /usr/bin directory is taken as backup in the /usr/local/backup location locally.
+Note: PASSPHRASE should be more than 8 characters.
+```
+root@demo:~# export PASSPHRASE=12345678
+root@demo:~# duplicity /usr/bin file:///usr/local/backup
+Local and Remote metadata are synchronized, no sync needed.
+Last full backup date: none
+No signatures found, switching to full backup.
+--------------[ Backup Statistics ]--------------
+StartTime 1661752359.77 (Mon Aug 29 05:52:39 2022)
+EndTime 1661752362.69 (Mon Aug 29 05:52:42 2022)
+ElapsedTime 2.92 (2.92 seconds)
+SourceFiles 582
+SourceFileSize 49793026 (47.5 MB)
+NewFiles 582
+NewFileSize 49793026 (47.5 MB)
+DeletedFiles 0
+ChangedFiles 0
+ChangedFileSize 0 (0 bytes)
+ChangedDeltaSize 0 (0 bytes)
+DeltaEntries 582
+RawDeltaSize 49779683 (47.5 MB)
+TotalDestinationSizeChange 21135788 (20.2 MB)
+Errors 0
+```
+Note: Two different data sets cannot be taken as backup into the same location. So make sure when a data set different from the previous is taken as backup, choose a different storage location.
+
+### Verification of data integrity before restore
+We can verify whatever changes are made to our latest file system on comparing with a latest backup taken like below: 
+```
+root@demo:/usr/bin# rm google-authenticator
+root@demo:~# duplicity verify --compare-data -v4 file:///usr/local/backup /usr/bin
+Last full backup date: Mon Aug 29 05:52:31 2022
+GnuPG passphrase for decryption:
+Difference found: File . has mtime Mon Aug 29 05:56:13 2022, expected Mon Aug 29 03:54:07 2022
+Difference found: File google-authenticator is missing
+Verify complete: 582 files compared, 2 differences found.
+```
+
+#### Data Restore
+The files can be restored only after integrity verification as shown above so that the user will specifically know which files or directories have to be restored due to an unwanted change or deletion in the file system.
+
+```
+root@demo:~# duplicity --file-to-restore google-authenticator file:///usr/local/backup /usr/bin/google-authenticator
+```
+Situation : If a directory containing 1000 files has around 250 files deleted and around 100 files modified. In that case it is not user friendly for the user to individually restore such a huge list of modified files to restore. Duplicity will not allow overwrite of remaining files in the directory so in that case the only possible solution is to delete the entire directory and restore it from the backup.
+
+The --file-to-restore switch should be removed if the user wants to restore a complete directory like shown below. 
+```
+root@demo:~# duplicity file:///usr/local/backup /usr/bin/
+```
+
+### Listing all the files in the backup
+To check all the contents of a backup do as shown below:
+```
+root@demo:~# duplicity list-current-files file:///usr/local/backup
+Local and Remote metadata are synchronized, no sync needed.
+Last full backup date: Mon Aug 29 05:52:31 2022
+Mon Aug 29 03:54:07 2022 .
+Mon Aug 29 03:52:37 2022 Mail
+Thu Sep 24 08:36:09 2020 [
+Thu Jan 20 20:10:35 2022 addpart
+Sun Jan 16 12:36:56 2022 aide
+Thu Jun 10 08:53:34 2021 apt
+Thu Jun 10 08:53:34 2021 apt-cache
+Thu Jun 10 08:53:34 2021 apt-cdrom
+Thu Jun 10 08:53:34 2021 apt-config
+...
+```
+
+# Remote Backup & Restore
+### Setting up the environment on remote server
+In the Local server, generate a pair of RSA keys that will be used for authentication during remote backup.
+```
+root@demo:~# ssh-keygen -t rsa -b 2048 -N "" -f /root/.ssh/id_rsa
+root@demo:~# ssh-copy-id -i ~/.ssh/id_rsa remoteuser@remoteserver-ip
+```
+
+Create a directory in remote server for keeping the backup files
+```
+root@demo:~# ssh remoteuser@remoteserver mkdir -p sftp_remote_bk_testing/first_backup
+```
+
+### Generation of GPG keys for encryption of data
+Create a file name gen-key-script and add the parameters like below:
+```
+root@demo:~# cat >foo <<EOF
+     Key-Type: DSA
+     Key-Length: 1024
+     Subkey-Type: ELG-E
+     Subkey-Length: 1024
+     Name-Real: username
+     Name-Email: usermailID
+     Expire-Date: 0
+     # passphrase should be more than 8 characters
+     Passphrase: 123456789
+EOF
+root@demo:~# gpg --batch --gen-key foo
+gpg: Generating a basic OpenPGP key
+gpg: key AE57464HFG4 marked as ultimately trusted
+gpg: revocation certificate stored as '/root/.gnupg/openpgp-revocs.d/E4C84C97B37BB24947E4522DA4B60C25738850E9.rev'
+```
+Use the encrypted key generated as above command and the encrypt key is AE57464HFG4 in the remote backup.
+
+### Remote Backup
+```
+root@demo:~# duplicity --encrypt-key "AE57464HFG4" /usr/bin sftp://remoteuser@remoteserver:/sftp_remote_bk_testing/first_backup
+```
+Find the backed up files in the remote location using the below command
+```
+root@demo:~# ssh remoteuser@remoteserver ls /home/remoteuser/sftp_remote_bk_testing/first_backup
+- duplicity-full.20220829T070950Z.manifest.gpg 
+- duplicity-full.20220829T070950Z.vol1.difftar.gpg  
+- duplicity-full-signatures.20220829T070950Z.sigtar.gpg
+```
+
+### Verification of integrity after recent backup in remote server
+```
+root@demo:~# duplicity verify --encrypt-key "AE57464HFG4" --compare-data  sftp://remoteuser@remoteserver:/sftp_remote_bk_testing/first_backup /usr/bin
+Local and Remote metadata are synchronized, no sync needed.5CAE2A504" --compare-data  sftp://remoteuser@remoteserver:/sftp_remote_bk_testing/first_backup /usr/bin
+Last full backup date: Mon Aug 29 07:09:50 2022
+GnuPG passphrase for decryption: 
+Difference found: File . has mtime Mon Aug 29 07:14:15 2022, expected Mon Aug 29 06:00:22 2022
+Difference found: File google-authenticator is missing
+Verify complete: 582 files compared, 2 differences found.
+```
+
+#### Restoration of data from remote server
+```
+root@demo:~# duplicity --file-to-restore google-authenticator sftp://remoteuser@remoteserver:/sftp_remote_bk_testing/first_backup /usr/bin/google-authenticator
+```
+
+# Automatic Backup scheduling using cron service
+To avoid operator overhead to regularly take system backup, it is necessary to automate the backup so that the data backup will be **up-to-date**.
+
+While automating this backup process, it is suggestible to use remote backup based on space issue. 
+
+Below a simple script which shall be added in the crontab to run on a daily basis.
+
+```
+#!/bin/bash
+echo "This will execute the backup on the first minute, first hour , every day"
+export PASSPHRASE=12345678
+duplicity --encrypt-key "AE57464HFG4" /usr/bin sftp://remoteuser@remoteserver:/sftp_remote_bk_testing/first_backup
+unset PASSPHRASE
+```
+
+In this demo script, the /usr/bin is being taken as a backup. The user can replace based on the requirement.
+The passphrase should also match the one assigned by the user during GPG keys creation.
+
+Save the file (daily_backup.sh) in a known path and start the cron service.
+```
+root@demo:~# systemctl start cron
+```
+Check the status of the service (expected to be in running state)
+```
+root@demo:~# systemctl status cron
+‚óŹ cron.service - Regular background program processing daemon
+     Loaded: loaded (/lib/systemd/system/cron.service; enabled; vendor preset: enabled)
+     Active: active (running) since Mon 2022-08-29 12:11:25 UTC; 12min ago
+```
+Open the crontab 
+```
+root@demo:~# crontab -e
+```
+Add the below line at the end of the crontab file if the user needs a backup on daily basis onto the remote server.
+```
+0 0 * * * /root/daily_backup.sh
+```
+After saving the crontab, the below message should be displayed.
+**crontab: installing new crontab**
+The user can verify this automation by checking the contents of the remote backup folder regularly.