Backup time and throttling

  • Hi,


    Over time when we went online with our hosting solution for Concrete5 Denmark, I was really happy to see how easy to setup and how smooth i-MSCP was running. It is no secret that Concrete5 CMS is a heavy lady when it comes to using resources. But still, both our servers is running flawlessly. But...


    Every night at midnight the backup starts on all domains. Not a huge problem when it came to one 5 to 10 domains, but now where we approach about 50 domains, the backup just takes the teeth out of both CPU and disks. (see attached graph). and it takes more than 4 hours where it is sometimes impossible to load a website, it simply times out.


    Is there not anything I can do to throttling down the use of resources when backing up?


    I think that I have read a thread where someone talked about taking one main backup and then a incremental backup. That would be an interesting approach!


    I also read that in 1.2 the backup system would be rebuild. Could you give some kind of time frame and an idea about what to expect?


    Best wishes
    /Michael

    Files

    Concrete5 Denmark - CMS til alle
    --------------------------
    Michael Jensen-Maar
    Concrete5 Danmark

    --------------------------

  • The build in backup solution is not recommended for so much domains (only if they are small). The feature needs a "new" implementation. I would use rdiff or something like that :)

  • I was worried that this would be the answer! But thanks!


    Concrete5 has its own backup solution (add-on) called "backup voodoo" that would be my choice for the future if I can't find any other solution. But it would be nice to have a automatic backup solution for i-MSCP that would be good enough for a hosting environment with more that 5 to 10 domains.


    I will look in to rdiff and see if I can figure out if that is the solution, but I think my level of knowledge of linux and backup routines is a little low when it comes to doing this my selves. It is simply too important that it is setup 100% correctly and I am not sure that I am able to do that.


    /Michael

    Concrete5 Denmark - CMS til alle
    --------------------------
    Michael Jensen-Maar
    Concrete5 Danmark

    --------------------------

  • Hello ;


    The problem with the i-MSCP backup solution is simple: The backup archives are reconstructed from scratch every nights and the compression level eats too much resources (CPU time, disk access time).


    I think that we can mitigate this problem by:

    • implementing an incremental backup
    • lowering the compression level
    • scheduling backup of different customers separately (per domain) at different time using cron
    • Allowing the admin to set the max size of files to backup
    • Lowering the resources consumption using the nice and ionice commands


    If you agreed with that, I can do that for the version 1.1.13

    badge.php?id=1239063037&bid=2518&key=1747635596&format=png&z=547451206

  • Hi Laurent,


    That would probably work, but I would like to know how you do it to day. I just did some tests with rsync, and that seem to work as far as I could make it from the command line. The nice thing about this is that I was able to rsync to a QNAP over the internet through SSH from the command line. And as I understand rsync will do incremental backup too. This is my first try with rsync, so I am not sure how to make this work securely.


    But the things that you wrote sounds great. Especially the incremental idea. That could make a huge difference.


    The cron schedule is something I have been wating on for a long time. Concrete5 can produce job url's to use that way. And if we then use Concrete5 add-on backup voodoo that could be used too, and in that way we would have multiply choices to implement backup.


    I am not sure how much the compression level would give, but worth a try!


    The option for the Admin to set max size of files - It would of course be an option to consider, but then we would not have a full backup...


    Lower te resource consumption - That could make a lot of sense. If we could make the backup use less processor that would be nise. But the problem is not the CPU's alone - The disc is running 100% through the backup, and I think that is the main issue.


    But I am up for the discussion and I will happily try things out on our develop server (it too have a lot of domains).


    At last I am happy to hear that this is something that you take seriously...


    /Michael

    Concrete5 Denmark - CMS til alle
    --------------------------
    Michael Jensen-Maar
    Concrete5 Danmark

    --------------------------

  • @MGAV


    When I say "schedule backup task via cron", this is on the engine side and therefore, I don't really understand what you mean here by <<< Concrete5 can produce job url's to use that way >>>


    About the IO operations

    The nice command allow to mitigate the CPU time consumption while the ionice command (see it as IO for input/output operations) allows to mitigate the disk access time ;)

    badge.php?id=1239063037&bid=2518&key=1747635596&format=png&z=547451206

  • Re:


    More info about the ionice command: http://linux.die.net/man/1/ionice

    badge.php?id=1239063037&bid=2518&key=1747635596&format=png&z=547451206

  • Ohh, I understand, my bad...


    I am talking about concrete5 as I do normally - I assumed you knew about the scheduled jobs in concrete5.


    I will try to explain. concrete5 has a job scheduling system (see first screen shot).


    It can produce cron jobs (see second screen shot).


    About ionice - I will try to put my mind in to that...

    Files

    Concrete5 Denmark - CMS til alle
    --------------------------
    Michael Jensen-Maar
    Concrete5 Danmark

    --------------------------

  • I use the i-mscp backup only for small databases. For large databases I use a separate script that only dumps the db. After all of this and to have a backup also of the files, not only databases, I use a rsync script that copy everything I need to restore a server in case of failure, on a dedicated backup server.
    Also the rsync is eating a lot of IO/CPU so the solution was to throttle the bandwidth and to set a lower priority on the rsync.


    To set a lower priority, in crontab I run the rsync script with " /usr/bin/nice -n 19 "
    To throttle the bandwidth used by rsync and set a lower encryption via ssh, " --bwlimit=6000 --recursive -e 'ssh -o Ciphers=arcfour' "

  • @Delta04


    The subject here is the backup solution as provided by i-MSCP, not your own solution :cursing: Please, stay in the subject (stickly) ;)


    BTW: Yes, I know. :D

    badge.php?id=1239063037&bid=2518&key=1747635596&format=png&z=547451206