psinib - Perl Snapshot Is Not Incremental Backup
|
Download perl script and install it anywhere on your system.
Includes this documentation page if you invoke it with perldoc
psinib.pl.
psinib.pl 25 Kb |
|
Optional subjects file for --email option
subjects.txt 539 bytes |
|
Whole archive with ChangeLog, TODO and example mountscript
psinib-1.0-rc3.tar.gz 12 Kb |
| Latest source is always available from Subversion repository |
Index
- NAME
- SYNOPSIS
- DESCRIPTION
- CONFIGURATION
- HACKS, TRICKS, BUGS and LIMITATIONS
- AUTHOR
- LICENSE
NAME
psinib - Perl Snapshot Is Not Incremental Backup
SYNOPSIS
./psinib.pl [OPTION]... [mount script]
DESCRIPTION
Backup samba or Windows shares using LinNeighborhood mount script as configuration
Option can be one of more of following:
--backupdest=dir-
Specify backup destination directory (default is /backup/)
--noping-
Don't use ping to check if host is up (default is to use TCP SYN to CIFS port)
--verbose -v-
Increase verbosity level. Default is 1 which prints moderate amount of data on STDOUT and STDERR.
--quiet -q-
Decrease verbosity level
--email=email@domain-
Send e-mails instead of dumping errors to STDERR. Useful for cron jobs.
--max_share_size=50000-
Specify maximum size of share to backup (in Kb). This example will skip backuping shares which have more than 50Mb. By default this option is disabled.
--max_file_size=5000-
Specify maximum size of file to backup (in Kb). This example will limit file size to 5Mb. By default this option is disabled.
--min_free_space=0-
Specify minumum free space on destination file system to start backup. If space is not availabe, backup will abort. By default, 100Mb of free space is required on destination. You can set this value to 0 to disable this check.
--diff-
Show compact diff-like output for changed files
--only=part_of_share_name-
Backup just shares which match
part_of_share_namein this run. --help -h-
Display usage page.
Idea of this script is to produce tool which will be easy enough to use for any Linux administrator which has a task to backup Windows workstations or samba shares. Backup is done to central disk space on Linux server running psinib. It's organized in multiple directories named after:
- * server which is sharing files to be backed up
- * name of share on server
- * dated directory named like standard ISO date format (YYYYMMDD).
In each dated directory you will find snapshot of all files on exported share on that particular date.
You can also use symlink latest which will lead you to last completed backup. After that you can use some other backup software to transfer snapshot to tape, CD-ROM or some other media, or just re-export it via samba so that users can do restore of files on specific date themselves.
Design considerations
Since taking of share snapshot every day requires a lot of disk space and network bandwidth, psinib uses several techniques to keep disk usage and network traffic at acceptable level:
- - usage of hard-links to provide same files in each snapshot (as opposed to have multiple copies of same file)
- - usage of file size, atime and mtime to find changes of files without transferring whole file over network (just share browsing is transfered over network)
- - usage of
.md5sumfiles (compatible with command-line utilitymd5sum) to keep file between snapshots hard-linked
CONFIGURATION
Configuration file format is simple:
smbmount //WIN_BOX/data /home/dpavlin/mnt/WIN_BOX/data/ -o username=dpavlin%password,fmask=644,dmask=755,uid=1000,gid=1000,debug=0,workgroup=HOME
This file is produced by LinNeighborhood http://www.bnro.de/~schmidjo/
utility. So to make backup of your Windows share, it's enough just to mount
it using LinNeighborhood, and use Export Mountscript to produce
mountscript which is at the same time configuration file for psinib.
HACKS, TRICKS, BUGS and LIMITATIONS
This chapter will have all content that doesn't fit anywhere else.
Can snapshots be more frequent than daily?
There is not real reason why you can't take snapshot more often than once a day. Actually, if you are using psinib to backup Windows workstations you already know that they tend to come-and-go during the day (reboots probably ;-), so running psinib several times a day increases your chance of having up-to-date backup (psinib will not make multiple snapshots for same day, nor will it update snapshot for current day if it already exists).
However, changing psinib to produce snapshots which are, for example, hourly
is a simple change of $DIR_TIME_FMT which is currently set to
'%Y%m%d' (see strftime documentation for explanation of that
format). If you change that to '%Y%m%d-%H you can have hourly snapshots
(if your network is fast enough, that is...). Also, some of messages in
program will sound strange, but other than that it should work.
You have been warned.
Do I really need to share every directory which I want to snapshot?
Won't I run out of disk space?
Of course you will... Snapshots and logfiles will eventually fill-up your disk. However, you can do two things to stop that:
Clean snapshot older than x days
You can add following command to your root crontab:
find /backup/isis_backup -type d -mindepth 3 -maxdepth 3 -mtime +11 -exec rm -Rf {} \;
I assume that /backup/isis_backup is directory in which are your snapshots
and that you don't want to keep snapshots older than 11 days (that's
-mtime +11 part of command).
Rotate your logs
I will leave that to you. I relay on GNU/Debian's logrotate to do it for me.
What are YYYYMMDD.partial directories?
If there isn't latest symlink in snapshot directory, it's pretty safe to assume that previous backup from that day failed. So, that directory will be renamed to YYYYMMDD.partial and snapshot will be performed again, linking same files (other alternative would be to erase that dir and find second-oldest directory, but this seemed like more correct approach).
I can't connect to any share
Aren't backups boring?
No! If you have subjects.txt in same directory as psinib.pl you can get
various funny subjects in your mail. They change over time as long as you
ignore your backup.
Why all those limit on backup share and files?
AUTHOR
Dobrica Pavlinusic <dpavlin@rot13.org>
LICENSE
This product is licensed under GNU Public License (GPL) v2 or later.
ChangeLog
2008-10-26 10:33:49 dpavlin r51
/trunk/psinib.pl: We never want to pull .md5sum from client back to server, because it should end up on client only if we restored our backup on it. And if it does, it's probably wrong...
2008-01-22 18:04:36 dpavlin r50
/trunk/psinib.pl: handle case where local copy of directory which is backuped is empty
2007-02-15 12:32:09 dpavlin r49
/trunk/psinib.pl: don't show ignored files when using --diff without --verbose
2007-02-14 15:59:40 dpavlin r48
/trunk/psinib.pl: added --only to backups just some shares, documented --diff
2007-02-14 02:00:23 dpavlin r47
/trunk/psinib.pl: don't re-create local directories over and over again
2007-02-14 02:00:06 dpavlin r46
/trunk/fdupes-ln.pl: minor fix
2007-02-14 01:55:36 dpavlin r45
/trunk/psinib.pl: make Perl::Critic not critical, fixed temporary name creation [1.0-rc3]
2007-02-14 01:42:48 dpavlin r44
/trunk/psinib.pl: added long forgot --diff option which output just compact changes
2007-02-14 01:30:09 dpavlin r43
/trunk/fdupes-ln.pl: make Perl::Critic happy
2007-02-14 01:29:56 dpavlin r42
/trunk/Makefile: critic target
2007-02-14 00:54:16 dpavlin r41
/trunk/fdupes-ln.pl: skip directories (fdupes would groak, so why let him?)
2007-02-13 23:19:53 dpavlin r40
/trunk/fdupes-ln.pl: proper pod docs
2007-02-13 23:19:40 dpavlin r39
/trunk/Makefile: use svk instead of svn
2007-02-13 23:12:27 dpavlin r38
/trunk/fdupes-ln.pl: helper script to recover from network errors (and this broken chain-of-hardlinks in some shares)
2007-02-13 22:22:11 dpavlin r37
/trunk/psinib.pl: be a bit more chatty at default verbosity 1, transfer files to temporary ones and rename them after succesfull transfer (instead of doing it in-place and possibly corrupting files because of network errors) [1.0-rc1]
2005-06-09 16:05:43 dpavlin r36
/trunk/psinib.pl: fix verbose
2004-07-08 13:00:57 dpavlin r35
/trunk/psinib.pl: checking for host_up is much more agressive now (so it will detect machines which have firewall enabled, but also allowed file sharing). It will also always use IP address when connecting to SMB share, with fallback to host if nmblookup didn't return IP
2004-06-30 13:23:01 dpavlin r34
/trunk/psinib.pl: There is a problem with List::Compare up to 0.30, so we require 0.30 or newer
2004-06-13 13:28:25 dpavlin r33
/trunk/psinib.pl: --backupdest now takes arguments (stupid bug) and typo fix
2004-06-13 13:24:31 dpavlin r32
/trunk/psinib.pl: bug fix: change file and share size to Kb before comparison
2004-06-11 23:26:43 dpavlin r31
/trunk/psinib.pl: Major improvement and fixups for version 0.9:
- added options --max_share_size, --max_file_size, --min_free_space and --help
- display all values in Kb
- enable warn and die before log is opened
- documentation fixes
2004-05-15 13:06:32 dpavlin r30
/trunk/psinib.pl: improved documentation
2004-05-14 10:02:40 dpavlin r29
/trunk/Makefile, /trunk/psinib.pl: release of version 0.8
2004-05-14 09:27:29 dpavlin r28
/trunk/psinib.pl: use rmtree instead of rmdir, and check if directory exists
2004-03-01 19:38:16 dpavlin r27
/trunk/psinib.pl: if home directory isn't writable (for e.g. if you are running psinib under backup user with Debian) it will fallback to /tmp directory (which is somewhat bad -- why whould you want to run under user without writable home dir?)
2003-11-08 01:46:53 dpavlin r26
/trunk/psinib.pl: don't warn on undef if there isn't subjects.txt file
2003-11-08 00:31:14 dpavlin r25
/trunk/subjects.txt, /trunk/psinib.pl: fixed division by zero if no valid mounts are found in mountscript added --email option to send e-mails (use Mail::Send) implemented cycling subjects (use Tie::File)
2003-10-27 19:07:32 dpavlin r24
/trunk/psinib.pl: remove non-valid latest links
2003-10-27 18:58:41 dpavlin r23
/trunk/psinib.pl: spit fatal errors on screen even if quiet (use -q -q to supress all output)
2003-10-26 14:04:17 dpavlin r22
/trunk/psinib.pl: verbose, quiet options and added documentation for command line options
2003-10-26 12:55:56 dpavlin r21
/trunk/psinib.pl: use tcp syn ping to check if host if up
2003-10-12 21:46:42 dpavlin r20
/trunk/psinib.pl: doc add-on
2003-10-12 18:47:35 dpavlin r19
/trunk/psinib.pl: command-line options
2003-10-12 17:44:21 dpavlin r18
/trunk/psinib.pl: remove stale entries in .md5sum
2003-10-12 16:13:38 dpavlin r17
/trunk/psinib.pl: better logging, report correct number of dirs, don't create local file if remote permission doesn't allow reading
2003-10-12 15:58:28 dpavlin r16
/trunk/psinib.pl: add support for ip=1.2.3.4 option which overrides nmblookup
2003-07-15 17:41:45 dpavlin r15
/trunk/psinib.pl: removed all debugging output
2003-07-15 17:40:32 dpavlin r14
/trunk/psinib.pl, /trunk/TODO: fixed bugs dealing with latest links and smb mounts which aren't available
2003-03-04 21:08:43 dpavlin r13
/trunk/psinib.pl, /trunk/TODO: some more warnings
2003-01-31 22:32:42 dpavlin r12
/trunk/TODO: allmost at version 0.9
2003-01-31 22:31:41 dpavlin r11
/trunk/psinib.pl: Lot of changes:
- ping using tcp connect to port 139
- save warn and die in log
- use newest directory as latest (or partial) if latest symlink doesn't exist
- actually CREATE latest symlink
2003-01-21 21:30:19 dpavlin r10
/trunk/psinib.pl: how to erase old snapshots
2003-01-21 19:50:47 dpavlin r9
/trunk/psinib.pl: append log entries to log
2003-01-04 15:59:14 dpavlin r8
/trunk/psinib.pl: creates .md5sum files (but still doesn't use them) better documentation
2003-01-04 13:34:38 dpavlin r7
/trunk/mountscript.example: example of mountscript used for configuration (created with LinNeighborhood)
2003-01-04 13:29:12 dpavlin r6
/trunk/TODO, /trunk/psinib.pl: start of POD documentation -- use 'perldoc psinib.pl' to read it
2003-01-04 12:14:54 dpavlin r5
/trunk/TODO, /trunk/psinib.pl: added locking using logfile
2004-05-14 09:02:52 unknown r4
/tags, /tags/r0: This commit was manufactured by cvs2svn to create tag 'r0'.
2003-01-04 11:43:18 dpavlin r3
: initial import of first working version
2003-01-04 11:43:18 unknown r2
/branches, /branches/DbP: This commit was manufactured by cvs2svn to create branch 'DbP'.
2003-01-04 11:42:55 dpavlin r1
/trunk/TODO, /trunk/psinib.pl, /trunk: Initial revision