Perfect Backup Solution with Amazon S3 and aws

Productivity Sauce
Amazon S3 provides unlimited storage at low prices, which makes it an ideal solution for storing backups. But to make use of it, you need a piece of software that can actually interact with Amazon S3: create buckets, list the contents of a bucket, upload and download files, etc. And aws, a simple command-line utility written in Perl, is the perfect tool for the job. You might wonder why not use a GUI-based application like Jungle Disk? For two simple reasons: as a CLI-based tool, aws is light on resources and it can be easily scripted.
Before you proceed, you should install the curl utility. On Ubuntu, you can do this using the sudo apt-get install curl command. Next, grab the latest version of the aws script:
curl timkay.com/aws/aws -o aws
Make it then executable and copy it to the /usr/bin directory:
chmod +x aws sudo cp ~/aws /usr/bin/
Create then an .awssecret file and open it in a text editor like nano:
nano .awssecret
Enter your Amazon AWS credentials (the Access Key ID and the Secret Access Key) as follows:
1B5JYHPQCXW13GWKHAG2 2GAHKWG3+1wxcqyhpj5b1Ggqc0TIxj21DKkidjfz
Save the file and change its permissions:
chmod 600 .awssecret
aws is now ready to go. To create a bucket for your backup use the aws mkdir command (replacing BUCKET with the actual name):
aws mkdir BUCKET
Next, create a tarball of the directory you want to back up using the tar tool:
tar -pvczf tmp/dir.tar.gz /path/to/dir
Finally, upload the created archive to the created bucket:
aws put BUCKET/dir.tar.gz /path/to/dir.tar.gz
The best part is that you don't have to do this manually every time you want to back up a certain directory. Here is a sample script that backs up photos stored on the local hard disk:
#!/bin/bash cd /home/user/ tar -pvczf Photos.tar.gz Photos aws put BUCKET/Photos.tar.gz Photos.tar.gz
Tweak the script to your liking, schedule it using cron, and your perfect Amazon S3-based backup solution is ready to go.
Comments
comments powered by DisqusSubscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you've found an article to be beneficial.
News
-
Red Hat Migrates RHEL from Xorg to Wayland
If you've been wondering when Xorg will finally be a thing of the past, wonder no more, as Red Hat has made it clear.
-
PipeWire 1.0 Officially Released
PipeWire was created to take the place of the oft-troubled PulseAudio and has finally reached the 1.0 status as a major update with plenty of improvements and the usual bug fixes.
-
Rocky Linux 9.3 Available for Download
The latest version of the RHEL alternative is now available and brings back cloud and container images for ppc64le along with plenty of new features and fixes.
-
Ubuntu Budgie Shifts How to Tackle Wayland
Ubuntu Budgie has yet to make the switch to Wayland but with a change in approaches, they're finally on track to making it happen.
-
TUXEDO's New Ultraportable Linux Workstation Released
The TUXEDO Pulse 14 blends portability with power, thanks to the AMD Ryzen 7 7840HS CPU.
-
AlmaLinux Will No Longer Be "Just Another RHEL Clone"
With the release of AlmaLinux 9.3, the distribution will be built entirely from upstream sources.
-
elementary OS 8 Has a Big Surprise in Store
When elementary OS 8 finally arrives, it will not only be based on Ubuntu 24.04 but it will also default to Wayland for better performance and security.
-
OpenELA Releases Enterprise Linux Source Code
With Red Hat restricting the source for RHEL, it was only a matter of time before those who depended on that source struck out on their own.
-
StripedFly Malware Hiding in Plain Sight as a Cryptocurrency Miner
A rather deceptive piece of malware has infected 1 million Windows and Linux hosts since 2017.
-
Experimental Wayland Support Planned for Linux Mint 21.3
As with most Linux distributions, the migration to Wayland is in full force. While some distributions have already made the move, Linux Mint has been a bit slower to do so.
New Download Link
Bucket Explorer
You can also try User Interface , which is called Bucket Explorer. Technically, its probably the best solution out there to transfer files to S3 and when you want to be in control, as it has easy to use UI, which is built on top of the robust JetS3t API.
You can use Bucket Explorer as a simple FTP tool or a backup tool for Amazon S3 or you can use it to:
1. Browse buckets and the files stored at Amazon S3.
2. Upload and download files in multiparts, to and from Amazon S3 buckets.
and many more..
Bucket Explorer works on every OS where Java is supported. It uses Amazon’s ETag and its own SHA-1 hash combination to make sure that a file is never transferred again to Amazon S3 if is it not changed, to save the bandwidth costs and time.
http://www.bucketexplorer.com/
Not found
404 Error File Not Found
Amazon S3 Storage
You can use Bucket Explorer as a simple FTP tool or a backup tool for Amazon S3 or you can use it to:
It facilitates many features.
Bucket Explorer works on every OS where Java is supported. It uses Amazon's ETag and its own SHA-1 hash combination to make sure that a file is never transferred again to Amazon S3 if is it not changed, to save the bandwidth costs and time.
Other references
http://jackal777.wordpress....s-a-local-filesystem-in-rhel5/
http://jackal777.wordpress....22/cpanel-backup-to-amazon-s3/
Gui for Linux
Amazon S3
1. Store encrypted backups in S3. Unlimited backup storage. Off-site. No changing tapes. Very handy.
2. We were suddenly and unexpectedly hit with a massive amount of traffic downloading product images from our server. Our bandwidth with our ISP was maxing out and we were looking at very scary (around ten thousand dollars!) overage fees. We uploaded the product images to a bucket named images.ourcompany.com and CNAME'd the record in our DNS to point at S3. Instantly Amazon was virtual hosting images.ourcompany.com for us and we ended up serving it all much faster with no extra load on our servers for only $25 for the whole incident. Customers got the product images, sales skyrocketed during this incident, and I was commended for having come up with a wonderful solution (which is now a permanent arrangement) in a pinch. Yeay!
I don't work for Amazon or benefit in any way from endorsing them. I'm just a happy corporate customer.
firefox
SpiderOak
yaaaaawn
Well... no versioning... no diff...