I use bitpocket as a DIY 2-way synchronization system. It serves both for backup purposes as well as keeping multiple systems in sync.
At its core, the solution leverages rsync in intricate ways; nothing of cloud-based or closed software.
The open-source, transparent, home brewed system does everything you would expect of a SaaS cloud-based setup, excepting the sharing and collaboration of data. And this makes it so much smaller and simpler. The sharing would effectively render it a complex pseudo-CMS system, imposing a series of infrastructural constraints.
Bitpocket conducts backups of any data overwritten during a synchronization, which can take place on the slaves and/or the master. It even integrates with git for incremental backup functionality at very little expense (I’ve examined the source).
It also supports include/exclude lists based entirely on rsync filter rules, making this a fully selective synchronization tool.
Here I demonstrate a small segment of such an exclude list I maintain within the .bitpocket subfolder of my home directory. This effectively manages all data relative to $HOME I wish synchronized with the backup server.
- .git/
- temp*
- *.log
+ /.config/
+ /.config/bash/
- /.config/bash/history
+ /.config/bash/**
+ /.config/calcurse/***
+ /.config/git/***
+ /.config/inputrc
+ /.config/msmtp/
+ /.config/msmtp/config.*
+ mystyle.sty
- *
The ‘-’ prefixes indicate the data to exclude during the sync, and the ‘+’ the included.
Note the - \* at the bottom, this being the key to exclude everything not explicitly included above. Also note the excludes at the top. These dictate the types of files/folders to always exclude from any otherwise included directories below. The penultimate + mystyle.sty matches this included file within any included folder.
A single asterisk indicates all file patterns within the folder, whereas a double ** indicates all files and subfolders, recursively.
Per the rsync rules, all subfolders not included in the recursion must explicitly be included one by one. The exception is the /folder/*** syntax, which functions equivalently to /folder/ proceeded by /folder/**. (The rsync man page documents these intricacies.)
Bitpocket is not without odd bugs/side-effects, but they hardly impact the overall value.
For instance, it may require more than one sync iteration for data to synchronize between systems as you would expect. Also, files reverted to their immediately previous version between successive iterations sometimes confuse the tool and lead to a sync in the opposite direction.
However, these cases are rare, and no data ever disappears as long as you maintain the backup feature enabled. I’ve extensively monitored the verbose sync operations to insure of this.
Questions, comments? Connect.