Multiple backups

**Cool** · Jan 7th 2013

DragonZX
there is also the question if the backupspace should be included in diskspace or not, should it be a free service or a paid service...and so on

gabesz86
anyone here is understanding what you like to achieve...but that is what you want.

i-mscp dev´s allways have to find a balanced way wich will fit for most of theire users not only for one...and just that is not allways easy.

but to help you further with a quick solution for your needs you just have to multiple
my "dirty solution" to the amount of days you like. then inside the crons you set wich cron should run on wich day.
you also have to do a chown/chmod to the backup directories(can eaven be done inside the backup-script) so that they are readable/changeable by user.

so at this stage you have your backup-strategie.
then you have to inform the user(s) that they have to rename the related backup-folder to the std. backup-folder(std. backupfolder has to be renamed first)
so the user can restore any backup wich exists

in this way you have what you want in about 15min.

**Nuxwin** · Jan 7th 2013

Quote from Cool

DragonZX
there is also the question if the backupspace should be included in diskspace or not, should it be a free service or a paid service...and so on

gabesz86
anyone here is understanding what you like to achieve...but that is what you want.

i-mscp dev´s allways have to find a balanced way wich will fit for most of theire users not only for one...and just that is not allways easy.

but to help you further with a quick solution for your needs you just have to multiple
my "dirty solution" to the amount of days you like. then inside the crons you set wich cron should run on wich day.
you also have to do a chown/chmod to the backup directories(can eaven be done inside the backup-script) so that they are readable/changeable by user.

so at this stage you have your backup-strategie.
then you have to inform the user(s) that they have to rename the related backup-folder to the std. backup-folder(std. backupfolder has to be renamed first)
so the user can restore any backup wich exists

in this way you have what you want in about 15min.

Display More

Hello ;

And what about a new feature for backup on demand?

I see something like this:

1. Admin can allow reseller to activate backup on demand option for their resellers
2. Reseller can activate backup on demand for their customer and specifies how many demand they can trigger per day?

Of course, we still have daily backup (one per day).

It's easy to add so ?

Thanks for using i-MSCP.

**DragonZX** · Jan 7th 2013

Quote from Nuxwin

Hello ;

And what about a new feature for backup on demand?

I see something like this:

1. Admin can allow reseller to activate backup on demand option for their resellers
2. Reseller can activate backup on demand for their customer and specifies how many demand they can trigger per day?

Of course, we still have daily backup (one per day).

It's easy to add so ?

Thanks for using i-MSCP.

Display More

Yes, backup on demand is not so hard work, but now it can make more questions, then answers.
- Client must choose what to restore (GUI changes)
- Reseller or admin must choose how much disk space it wil need to make for backups (it's better to know how many disk space use only site, for example some of my clienst stored there about 40 GB of data).
In this case we must rewie the backup engine, don't hurry. I think it must be only in 1.2.0

**Nuxwin** · Jan 7th 2013

Quote from DragonZX

Yes, backup on demand is not so hard work, but now it can make more questions, then answers.
- Client must choose what to restore (GUI changes)
- Reseller or admin must choose how much disk space it wil need to make for backups (it's better to know how many disk space use only site, for example some of my clienst stored there about 40 GB of data).
In this case we must rewie the backup engine, don't hurry. I think it must be only in 1.2.0

Backup on demand button will only trigger the backup script for customer.
Disk space for user should be based on customer disk space limit (no space for backup then, no backup). Now, if your user are storing about 40GB, it's your problem. You must fire them about your rules or downgrade their space limit to avoid such thing.

**DragonZX** · Jan 7th 2013

Quote from Nuxwin

Backup on demand button will only trigger the backup script for customer.
Disk space for user should be based on customer disk space limit (no space for backup then, no backup). Now, if your user are storing about 40GB, it's your problem. You must fire them about your rules or downgrade their space limit to avoid such thing.

But it may cause more bugs first of all. It's only a wishlist. I understand what you want, but I disagree with you. First of all we must concentrate on beta testing and fixing existing bugs, not to open new bugs to your own head. It will provide us to release 1.1.0 ASAP.

Also, for example I said about a problem, that may cause a Critical bug. I have only 2-3 such clients, but some users may have >30 such clients, and they may does't know about it. Don't hurry, we need a review. Also it may cause a server overload.

**Nuxwin** · Jan 7th 2013

Quote from DragonZX

But it may cause more bugs first of all. It's only a wishlist. I understand what you want, but I disagree with you. First of all we must concentrate on beta testing and fixing existing bugs, not to open new bugs to your own head. It will provide us to release 1.1.0 ASAP.

Also, for example I said about a problem, that may cause a Critical bug. I have only 2-3 such clients, but some users may have >30 such clients, and they may does't know about it. Don't hurry, we need a review. Also it may cause a server overload.

Re ;

Who has says that it will be implemented in 1.1.0 ? Also, you are talking about a critical bug without explain your thinking. Please, take care of my poor head.

Thanks.

**anarking** · Jan 8th 2013

i implemented this via cron. created "archive" folder in the backup directory, and the cron moved/renamed the backup files to the archive folder for people that requested archiving. this is why the main backup script had to be changed a bit for me so that it did not delete everything in backup directory.

i do still have plans to completely overhaul the backup system. i cancelled the idea of uncompressed tar + rsync because pbzip2 is much faster now.

future, i am thinking of using Duplicity. it is compressed with rsync, and incrementals, so it can archive many days with just the incremental backup sizes. it also has very simple commands for restore (files or db or both), that should be easy to initiate from i-MSCP, as well as create a new backup-on-demand.

new database entries and backup archiving option for how many days to keep, with duplicity referencing that for backup task.

this is the best option so far, that I have found. it is a good amount of work I have not gotten around to yet... been working on infrastructure and trying to figure out how to manage i-MSCP while it's not actually multi-server yet and i need it to be haha.

but Duplicity seems to be the best answer for the future...

Also, I think backups should count towards disk quota. or perhaps have its own disk quota, that is most preferable.

**gabesz86** · Jan 9th 2013

anarking can u share me how can you do it? I want to similar your solution. Or myabe i will make it with cron jobs if anyone help me.

Quote from Nuxwin

1. Admin can allow reseller to activate backup on demand option for their resellers
2. Reseller can activate backup on demand for their customer and specifies how many demand they can trigger per day?

Of course, we still have daily backup (one per day).

Nuxwin I want to daily backup too, I just want to keep them couple days like the i-MSCP config backups. If i-MSCP config backup system know what I want, I don't understand some people why told me "this is a big job". Ok I know some of my Idea a part of a bigger job, but NOW I will be happy if I keep daily backup more than 24hour. I have no time to save every customer backup data to my computer evry day. I think easier keep backups more than 24 hour. If need more space, I will growing they disk limit.

cbxk1xg · Feb 26th 2013

I would even push this subject further by simply asking: Why is there no backup of the mail boxes?! Imho this is a mandatory feature.

An why am I only able to choose a whole domain for backups? It would be much better to make an option for every sub-domain.
I'm hosting a small open source software download site and I don't want to gzip a software repository with 200 isos every night, just because it's in /var/www/virtual/domain/subdomain/htdocs.

cbxk1xg · Feb 27th 2013

I wrote a patched version of the imscp backup script. I modified the script to keep the backups for seven days.

The original file is located at: /var/www/imscp/engine/backup/imscp-backup-all
Make a backup of the file, before you patch it.

Perl

#!/usr/bin/perl
# i-MSCP - internet Multi Server Control Panel
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
#
# @category i-MSCP
# @author Daniel Andreca <sci2tech@gmail.com>
# @link http://i-mscp.net i-MSCP Home Site
# @license http://www.gnu.org/licenses/gpl-2.0.html GPL v2
use strict;
use warnings;
use FindBin;
use lib "$FindBin::Bin/..";
use lib "$FindBin::Bin/../PerlLib";
use iMSCP::Debug;
use iMSCP::Boot;
use iMSCP::Execute;
use iMSCP::Mail;
use iMSCP::Dir;
use Data::Dumper;
newDebug('imscp-backup-all.log');
our $command = {
pbzip2 => {extension => 'bz2', command => 'CMD_PBZIP'},
bzip2 => {extension => 'bz2', command => 'CMD_BZIP'},
gzip => {extension => 'gz', command => 'CMD_GZIP'},
lzma => {extension => 'lzma', command => 'CMD_LZMA'},
xz => {extension => 'xz', command => 'CMD_XZ'}
};
sub backup_start_up {
umask(027);
iMSCP::Boot->new()->init({nolock => 'yes'});
0;
}
sub backup{
my $rs = 0;
my ($stdout, $stderr);
if ($main::imscpConfig{'BACKUP_DOMAINS'} ne 'yes') {
return 0;
}
unless ($main::imscpConfig{ZIP} =~ /^(pbzip2|bzip2|gzip|lzma|xz)$/i) {
my $msg = "Backup algorithm not supported: $main::imscpConfig{ZIP}!";
error($msg);
return 1;
}
my $sql = "
SELECT
*
FROM
`domain` AS t1,
`admin` AS t2
WHERE
t1.domain_status = 'ok'
AND
t1.domain_admin_id = t2.admin_id
AND
t1.allowbackup != 'no'
";
my $rdata = iMSCP::Database->new()->factory()->doQuery('domain_name', $sql);
if (ref $rdata ne 'HASH'){
error("$rdata");
$rs = 1;
} else {
for (keys %$rdata) {
iMSCP::Boot->new()->lock();
if ($rdata->{$_}->{allowbackup} !~ m/^no$/i) {
my $hDir = $main::imscpConfig{USER_HOME_DIR}."/$_";
my $bkpDir = "$hDir/backups";
my $rv |= iMSCP::Dir->new( dirname => $bkpDir)->make({
user => $main::imscpConfig{ROOT_USER},
group => $main::imscpConfig{ROOT_GROUP},
mode => 0755
});
$rs |= $rv;
if($rv){
iMSCP::Boot->new()->unlock();
next ;
}
# cbxk1xg patch to save backups for 7 days.
$rs |= execute("find $bkpDir/* -maxdepth 0 -type f -mtime +7 -print | xargs -r /bin/rm -v", \$stdout, \$stderr);
# $rs |= execute("$main::imscpConfig{'CMD_RM'} -rf $bkpDir/*", \$stdout, \$stderr);
debug($stdout) if $stdout;
error($stderr) if $stderr;
use POSIX qw(strftime);
my $ext = $command->{lc($main::imscpConfig{ZIP})}->{extension};
my $date = strftime "%Y.%m.%d-%H-%M", localtime;
if ($rdata->{$_}->{allowbackup} =~ m/full|dmn/i) {
my @cmd = (
lc($main::imscpConfig{CMD_TAR}),
"--create --directory=$hDir",
"--use-compress-program=" . lc($main::imscpConfig{ZIP}),
"--file=$bkpDir/$_-backup-$date.tar.$ext",
"--exclude=logs",
"--exclude=phptmp",
"--exclude=backups",
"--exclude=domain_disable_page",
". "
);
my $rs |= execute(join(' ', @cmd), \$stdout, \$stderr);
debug($stdout) if $stdout;
error($stderr) if $stderr;
my $file = iMSCP::File->new(filename => "$bkpDir/$_-backup-$date.tar.$ext");
$rs |= $file->mode(0666);
$rs |= $file->owner($main::imscpConfig{ROOT_USER}, $main::imscpConfig{ROOT_GROUP});
}
if (
$rdata->{$_}->{allowbackup} =~ m/full|sql/i
&&
iMSCP::Database->new()->factory()->can('dumpdb')
) {
$sql = "SELECT `sqld_id`, `sqld_name` FROM `sql_database` WHERE `domain_id` = ?";
my $dbData = iMSCP::Database->new()->factory()->doQuery('sqld_name', $sql, $rdata->{$_}->{domain_id});
if (ref $dbData ne 'HASH'){
error("$dbData");
$rs |= 1;
iMSCP::Boot->new()->unlock();
next;
}
for (keys %$dbData) {
my $date = strftime "%Y.%m.%d-%H-%M", localtime; # cbxk1xg patch to include the date in the file name
my $cmd = $main::imscpConfig{$command->{$main::imscpConfig{ZIP}}->{command}}." --force \'$bkpDir/$_-backup-$date.sql\'";
$rs |= iMSCP::Database->new()->factory()->dumpdb($_, "$bkpDir/$_-backup-$date.sql");
my $file = iMSCP::File->new(filename => "$bkpDir/$_-backup-$date.sql");
$rs |= $file->mode(0666);
$rs |= $file->owner($main::imscpConfig{ROOT_USER}, $main::imscpConfig{ROOT_GROUP});
$rs |= execute($cmd, \$stdout, \$stderr);
debug("$stdout") if $stdout;
error("$stderr") if $stderr;
error("Can not dump $_") if (!$stderr && $rs);
}
}
}
iMSCP::Boot->new()->unlock();
}
}
my @warnings = getMessageByType('WARNING');
my @errors = getMessageByType('ERROR');
my @all = (@warnings, @errors);
iMSCP::Mail->new()->errmsg("@all") if(scalar @all > 0 );
$rs;
}
my $rs = 0;
exit 1 if backup_start_up();
$rs |= backup();
exit $rs;

Display More

Multiple backups

Share

Daily visitors