constcyr
/
rsyncbackup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Browse Source

up

master
Cyril CONSTANTIN 4 weeks ago
parent
6f149add3c
commit
72025251ec
8 changed files with 294 additions and 1 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 16
      README.md
  2. 97
      backup_strategy.md
  3. 93
      banana_push_pull.sh
  4. 28
      export_bookmarks.sh
  5. 23
      git_prepare_and_push.sh
  6. 26
      mtp_put_directory.sh
  7. 6
      rsync_usb128go.sh
  8. 6
      rsync_usb64go.sh

16
README.md
Unescape View File

@ -1,3 +1,17 @@ @@ -1,3 +1,17 @@
# rsyncbackup
To backup and synchronize directories with redondancy on multiple support type (nas,computers,usb,cloud) by the most automated way. Push or pull with rsync.
To backup and synchronize directories with redondancy on multiple support type (nas,computers,usb,cloud) by the most automated way. Push or pull with rsync.
## Requirements
- /datas should contain P1 P2 Pxx with the same user
- connection with public/private keys (without password)
## banana_push_pull.sh
To synchronize datas between computers. Hosts should be available by ssh resolver (dns or .ssh/config) and rsync should be available installed on two sides.
## rsync_usb*.sh_
To sync mounted keys.
Copy script to key directory and execute it.

97
backup_strategy.md
Unescape View File

@ -0,0 +1,97 @@ @@ -0,0 +1,97 @@
# stratégie de classement et de sauvegarde des documents
# type de documents
* critères de tri
- taille (Ko/Mo/Go)
- durée de rétention (vie/1y/5y/10y)
- lisible en console
- oui:
- txt
- audio
- ods
- keepass
- non:
- video
- possible:
- pdf
- format tolérant les pertes
- oui (gravable):
- audio
- video
- non:
- archive (zip/tar/xz/7z/...)
- pdf
- convertible avec perte sous un format archivé
- oui:
- pdf
- non:
-
|-------------------|-----------|-------|-------|--------|--------|-----|-----|-----|----------|----------|-----|--------------|
| type | extension | duree | perso | repro | taille | cdr | usb | cli | disaster | rotation | maj | compressible |
|-------------------|-----------|-------|-------|--------|--------|-----|-----|-----|----------|--------- |-----|--------------|
| keepass | kbdx | vie | oui | oui | small | non | oui | oui | oui | oui | oui | non |
| paie | pdf | vie | oui | non | small | oui | oui | oui | oui | non | oui | non |
| scans officiel | png/pdf | vie | oui | oui | small | oui | oui | non | oui | non | oui | non |
| releves banques | pdf | 10y | oui | oui | middle | oui | oui | oui | non | oui | oui | non |
| video youtube | mp4 | - | non | oui | big | non | non | non | non | non | non | non |
| video famille | mp4 | vie | oui | non | big | oui | non | non | oui | non | oui | non |
| video entourage | mp4 | vie | oui | non | big | oui | non | non | non | non | oui | non |
| video idiotes | mp4 | - | oui | oui | big | oui | non | non | non | non | oui | non |
| video osm | mp4 | - | oui | oui | big | non | non | non | non | non | maj | non |
| video artois | mp4 | - | oui | non | big | oui | non | non | non | non | non | non |
| video copie | mp4 | - | non | oui | big | non | non | non | non | non | non | non |
| video anime | mp4 | vie | non | non | big | oui | non | non | non | non | non | non |
| video spectacle | mp4 | 10y | non | non | big | oui | non | non | non | non | non | non |
| books pdf study | pdf | 10y | non | non | middle | oui | non | oui | non | non | non | non |
| books epub study | epub | 10y | non | non | middle | oui | non | oui | non | non | non | non |
| audiobooks | ogg | 10y | non | non | big | oui | non | oui | non | non | non | non |
| music originaux | mp3/wma | 10y | non | non | middle | oui | non | oui | non | non | non | non |
| music dump cd | flac/... | - | non | oui | middle | oui | non | oui | non | non | non | non |
| iso winx cds | iso.xz | 10y | oui | non | big | non | non | non | non | non | non | oui |
| | | | | | | | | | | | | |
| | | | | | | | | | | | | |
| | | | | | | | | | | | | |
| | | | | | | | | | | | | |
# matrice des prioritées
|-----|-------|-------|--------|----------|-----|
| num | duree | perso | taille | archives | cdr |
|-----|-------|-------|--------|----------|-----|
| P1 | vie | oui | faible | non | non |
| P2 | vie | oui | moyen | oui | non |
| P3 | vie | oui | eleve | oui | non |
| P4 | vie | oui | eleve | non | oui |
| P5 | vie | oui | moyen | non | non |
| P6 | 10y | non | | | oui |
| P7 | 10y | non | | | oui |
| P8 | - | | | | non |
| P9 | - | | | | oui |
|-----|-------|-------|--------|----------|-----|
# tailles disques cibles
|----------------|----------|----|----|----|----|----|-----|-----|-----|
| volume | taille | P1 | P2 | P3 | P4 | P5 | P6 | P7 | P8 |
|----------------|----------|----|----|----|----|----|-----|-----|-----|
| | cible | 4 | 4 | 8 | 16 | 32 | 64 | 128 | 256 |
|----------------|----------|----|----|----|----|----|-----|-----|-----|
| | agregats | | 8 | 16 | 32 | 64 | 128 | 256 | 512 |
|----------------|----------|----|----|----|----|----|-----|-----|-----|
| | replicas | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 |
|----------------|----------|----|----|----|----|----|-----|-----|-----|
| cle usb 4Go | 4G | x | | | | | | | |
| cle usb 8Go | 8G | x | x | | | | | | |
| cle usb 16Go | 16G | x | x | x | | | | | |
| cle usb 32Go | 32G | x | x | x | x | | | | |
| cle usb 64Go | 64G | x | x | x | x | x | | | |
| cle usb 128G | 128G | x | x | x | x | x | x | | |
| noisette home | 256G | x | x | x | x | x | x | x | |
| externe 500Go | 512G | x | x | x | x | x | x | x | x |
| vg seagate 1To | 950G | x | x | x | x | x | x | x | x |
|----------------|----------|----|----|----|----|----|-----|-----|-----|

93
banana_push_pull.sh
Unescape View File

@ -0,0 +1,93 @@ @@ -0,0 +1,93 @@
#!/bin/bash
action=$(basename $0)
declare -a dirlist
logfile=/tmp/push_pull.log
rsync_options="--archive --perms --compress --delete-during --exclude=*.git* --exclude=git.code-workspace"
case "$(hostname)" in
pistache|noix|noisette)
dirlist=( /datas/workspace /datas/git /datas/P1 /datas/P2 /datas/P3 /datas/P4 /datas/P5 /datas/P6 /datas/P7 /datas/rebuild )
;;
queensland)
dirlist=( /datas/workspace /datas/git /datas/P1 /datas/P2 /datas/P3 /datas/P4 /datas/P5 /datas/P6 )
;;
pecan)
dirlist=( /datas/workspace /datas/git /datas/P1 /datas/P2 /datas/P3 /datas/P4 /datas/P5 )
;;
eee)
dirlist=( /datas/workspace /datas/git /datas/P1 /datas/P2 /datas/P3 )
;;
mllse)
dirlist=( /datas/P1/KEEPASS /datas/P1/PARENTS /datas/P2/*_THUOT /datas/P6/*_THUOT )
;;
*)
dirlist=( /datas/worspace )
;;
esac
# dirlist=( /datas/P1 /datas/P2 /datas/P3 /datas/P4 /datas/P5 /datas/P6 /datas/P7 )
ask() {
USER_ANSWER="?";
while [ "x$USER_ANSWER" != "xno" -a "x$USER_ANSWER" != "xnon" -a "x$USER_ANSWER" != "xyes" -a "$USER_ANSWER" != "xoui" ] ; do
echo -n "$1 $2 $3 ? [yes/no] "
read -n 3 USER_ANSWER
echo "";
done
if [ "x$USER_ANSWER" == "xyes" -o "x$USER_ANSWER" == "xoui" ] ; then
return 0
else
return 1
fi
}
echo "Snapshot git repositories (without history)"
case "$(hostname)" in
noix)
[ -e ${logfile} ] && rm ${logfile}
rsync --dry-run ${rsync_options} --exclude=*.terraform* --out-format="%B %l %M %f" /datas/git/* /datas/P1/git/ |tee ${logfile}
if [ -s ${logfile} ]
then
if ask snapshot git repo /datas/git /datas/P1/git
then
rsync --verbose ${rsync_options} --exclude=*.terraform* /datas/git/* /datas/P1/git/
fi
fi
;;
esac
echo "List of files differences from master"
for directory in ${dirlist[@]}
do
echo "check ${directory}"
[ -w ${directory} ] || ( echo "${directory} is not writable" && exit 3 )
[ -e ${logfile} ] && rm ${logfile}
case "${action}" in
pull)
rsync --dry-run ${rsync_options} --out-format="%B %l %M %f" banana:${directory}/ ${directory} |tee ${logfile}
[ -s ${logfile} ] || continue
if ask rsync banana:${directory}/ ${directory}
then
echo "rsync --verbose ${rsync_options} banana:${directory}/ ${directory}"
rsync --verbose ${rsync_options} banana:${directory}/ ${directory}
fi
;;
push)
rsync --dry-run ${rsync_options} --out-format="%B %l %M %f" ${directory}/ banana:${directory} |tee ${logfile}
[ -s ${logfile} ] || continue
if ask rsync ${directory}/ banana:${directory}
then
echo "rsync --verbose ${rsync_options} ${directory}/ banana:${directory}"
rsync --verbose ${rsync_options} ${directory}/ banana:${directory}
fi
;;
*)
echo "command ${command} is unknown ... push or pull ??"
exit 2
;;
esac
done

28
export_bookmarks.sh
Unescape View File

@ -0,0 +1,28 @@ @@ -0,0 +1,28 @@
#!/bin/bash
backupdirectory=/datas/P1/BOOKMARKS
bookmarksexportdir=/datas/git/bookmarksexport
export_bookmark()
{
echo "export ${browser}"
browser=$1
lastfile=$(find ${backupdirectory} -name "2[0-9]*_${browser}_bookmarks.html" -print |sort -n | tail -n 1)
lastfilechecksum=$(sha512sum "${lastfile}" |cut -d' ' -f 1)
python3 ${bookmarksexportdir}/bookmark_exporter.py --browser "${browser}" --formats html --output /tmp --prefix bookmark
lastexport=$(find /tmp -name "bookmark_2[0-9_]*" -print 2>/dev/null |sort -n |tail -n 1)
lastexportchecksum=$(sha512sum "${lastexport}" |cut -d' ' -f 1)
echo "compare ${lastexportchecksum} == ${lastfilechecksum}"
if [ "${lastexportchecksum}" == "${lastfilechecksum}" ]
then
echo "${browser} bookmarks not modified"
else
echo mv -i "${lastexport}" ${backupdirectory}/"$(date +%Y%m%d%H%M%S)_${browser}_bookmarks.html"
fi
}
for browser in Firefox Chromium Chrome
do
export_bookmark ${browser}
sleep 3
done

23
git_prepare_and_push.sh
Unescape View File

@ -0,0 +1,23 @@ @@ -0,0 +1,23 @@
#!/bin/bash
source=/datas/git
destination=/datas/git.ccyr.eu
echo "check diff from ${source} to ${destination}"
[ -e /tmp/rsync_push.log ] && rm /tmp/rsync_push.log
rsync -anpz --delete-after --exclude=*.git* --out-format="%B %l %M %f" ${source}/ ${destination} |tee /tmp/rsync_push.log
[ -s /tmp/rsync_push.log ] && echo "to push to distant, copy following command : \
rsync -apvz --delete-after --exclude=*.git* --out-format=\"%B %l %M %f\" ${source}/ ${destination}"
commit_repository()
{
echo "check repository $1"
cd ${destination}/$1
git status
}
for repository in ansible computers financial genealogie
do
commit_repository ${repository}
done

26
mtp_put_directory.sh
Unescape View File

@ -0,0 +1,26 @@ @@ -0,0 +1,26 @@
#!/bin/bash
# execute script from gvfs directory
directory=$1
# directory target name
destination=$(basename "${directory}")
echo "liste des fichiers du dossier ${directory}"
for item in "${directory}"/*
do
filename=$(basename "${item}")
echo "examen du fichier ${filename}"
if [ -e "${destination}"/"${filename}" ]
then
echo "${item} already on"
else
echo "copie du fichier ${filename}"
echo "cp --verbose ${item} ${destination}/${filename}"
cp --verbose "${item}" "${destination}"/"${filename}"
fi
done
# vim: tabstop=4 expandtab shiftwidth=4 softtabstop=4 foldmethod=indent

6
rsync_usb128go.sh
Unescape View File

@ -0,0 +1,6 @@ @@ -0,0 +1,6 @@
#!/bin/bash
for directory in "P1" "P2" "P3" "P4" "P5" "P6"
do
rsync -avzp --delete-during /datas/${directory}/ ./${directory}/
done

6
rsync_usb64go.sh
Unescape View File

@ -0,0 +1,6 @@ @@ -0,0 +1,6 @@
#!/bin/bash
for directory in "P1" "P2" "P3" "P4" "P5"
do
rsync -avzp --delete-during /datas/${directory}/ ./${directory}/
done
Write Preview
Loading…
Cancel
Save