Monday, December 19, 2011

How to back up web accessible files using wget, and to remove files older than 30 days

1. Script to back up files over wget:


#!/bin/bash
#
# This script grabs all the my-server.alanlupsha.com/logs/ files and
# backs them up on the local server in /home/
#
# @author Alan Lupsha 12/19/2011
#

# keep track of the start time
STARTTIME="Started backup at: `date`"


#########################################################################
# You should change these entries
#
# make sure to create the directory first
# ex: "mkdir /home/lupsha/backups"
BACKUPDIR=/home/lupsha/backups

# save text to a temporary file
EMAILFILE=/tmp/email.txt

# update this
EMAILADDRESS=my-email-address@my-domain.com
#########################################################################

# clean up email file
rm "$EMAILFILE"

# save file in this directory
cd "$BACKUPDIR"

LOC="`pwd`"
if [ "$LOC" != "$BACKUPDIR" ]; then
echo "ERROR, I should be in $BACKUPDIR, but I'm not! Exiting...";
exit 1
fi

# download files
wget -e robots=off --recursive --no-directories \
--no-host-directories --level=1 --no-clobber \
http://my-server.alanlupsha.com/logs/


# save timestamp
echo "$STARTTIME" >> $EMAILFILE
echo "\r\n" >> $EMAILFILE
echo "Ending pass logs backup at `date`" >> $EMAILFILE

# send an email with the result
cat $EMAILFILE | mail -s "Ran pass logs backup" $EMAILADDRESS



2. Save the script and run it from a cron job, only week days, at 7:30 AM


sudo crontab -e

# run script every week day at 7:30AM
30 07 * * 1-5 /home/lupsha/backup-logs-script.sh



3. On the server where the files are stored, remove the files that are older than 30 days:


cd /var/www/html/myfiles
find . -type f -mtime +30 -exec rm -- {} \;

No comments:

Post a Comment