Hourly monitoring of system updates make system unusable
On our ISPConfig Machines we see high IO load every hour. I could track it down to ISPConfig's monitoring function where apt-get is called on each system to a full hour. So on a server with many virtual machines, all running ISPConfig, all these apt-get processes are executed at the same time! This consumes all available IO for a ew minutes.
I have checked the source code (Version 184.108.40.206) - there is no way to disable this feature. The only way is to remove the monitoring module at all.
How to fix it:
- make the interval or the execution time configurable
- add a (probably configurable) random delay for each execution
- maybe best solution: on a multi-node-setup use the master to coordinate the updates across multiple servers (e.g. by assigning them a "minute" where the update is checked (by spreading them over the full hour) or better by running the updates on one machine after the other.
On debian based systems there is cron-apt that basically does the same - have a look to it, there exactly this problem is solved by a random startup delay.