Special backup method: "Manifest creator" or "Delegated backup"
Note: I know there is an issue to add support for BorgBackup (#6202 (closed)) to ISPConfig and I must admit I came with the following idea as a workaround to use Borg to backup my ISPConfig setups. But bear with me to understand how this proposal could help "third party" integration.
When I started using ISPConfig I needed a way to backup my websites (both files and databases) using my own existing scripts but because ISPConfig has built-in various full (id. A to Z) methods for backuping the data it manages there was no way to integrate with other tools/scripts (and I understant why: it was not needed).
Put it simply the situation is:
- ISPConfig knows (using the users' settings/preferences):
- where are the data and how to access them
- how often it must be backuped (backups frequency)
- how long (backups retention)
- My backup scripts knows what to do with files and SQL tables (read, compress, de-duplicate, encrypt, send to remote storage, etc.)
From that, my idea is to make ISPConfig "tell" other systems (an existing well-known tool, a self made script, ...) what the user wants to backup, and hence delegate the backup.
So I suggest the creation of a backup method for both websites files and databases that does not backup, compress nor encrypt anything, it would just create a manifest of what to backup.
For the files of a website, the manifest file would provide:
- Website name (eg. for naming the backups)
- The backup interval (the frequency)
- Number of backup copies (the retention)
- The full/absolute path of the base directory to backup
- The list of paths to exclude (cf. the "Excluded Directories" setting) as full/absolute paths.
For the database, the manifest file would provide:
- Database name (eg. for naming the backups)
- The backup interval (the frequency)
- Number of backup copies (the retention)
- Credentials to connect to the database server (as the backup/read-only user)
The manifest files would be recreated by ISPConfig when backup settings (frequency, retention, paths, databases, credentials, exclusions, etc.) are changed.
Then ISPConfig work is done and it's up to the other system/script to do the job, the way it detects changes to manifest files is not ISPConfig's business.
Some blur zones (non-exhaustive list):
- Backup triggers: I choose to write the backup frequency in the manifest so the backup tool/script can be aware of this frequency and run accordingly (eg. re-schedule itself or run everyday but detect when was the last execution and skip if not needed yet). But I think ISPConfig could trigger the backup, by executing a well-known command (eg.
/usr/bin/ispconfig/delegate-backup.sh /path/to/one/manifest-file
). - The fact the manifest file will contains the credentials and could be read by other. So I was thinking ISPConfig could write the credentials only when backup must be run and let the backup tool/script delete it.