SVN Part 1

Subversion is another awesome piece of software for keeping files in sync across multiple computers. Not quite suited for backing up pictures or anything like that -- but hey, its great for big coding projects! Usually with big projects, there are different levels of access. For example, Bob's algorithm may be super secret and only he and Bill should be able to see the code for it. But, Barry may be logging in to the same SVN server and need access to other code. There's an app for that. I mean, there's a feature for that. Also, perhaps not everyone should have access to the server's file, or even an account on the server. Well, there's a feature for that, too. Anyway, here's the procedure/setup I came up with after lots of research/trial-and-(hopefully not catastrophic) error.

Make sure at least YOU can ssh into the server (or better yet, be sitting in front of it and skip the next few steps). Its handy to use an RSA key so you can log in automatically and don't have to type your password every time you want to execute a subversion command. So:

sh-keygen -t rsa

will generate you a key. Then:

rsync -a ~/.ssh/id_rsa.pub server:~/.ssh/

Before we switch over to the server, if you're using a non-standard port, in /etc/ssh/ssh_config, add:

Host [server]
Port [port #]
Host * [this line should be there already, it just tells you where to put the 2 preceeding lines]

Next, ssh over to the server and:

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
rm ~/.ssh/id_rsa.pub

The >> instead of just > puts the key at the END of the file instead of overwriting, so if you already have a computer set up in your authorized_keys file, this will preserve that setup.

Now you should be able to log in with no password using ssh [server].

Part 2 coming soon! :P

APGT Linux Commands, Editors, and Shell Programming

The rsync scipts will be in the next edition of A Practical Guide to Linux Commands, Editors, and Shell Programming! The author and I go way back... ;)

Anyway, its (again) a handy utility. My roommate used to play all 90's music (and not the good stuff) on his Mac. The rsync utility comes default in Mac OS X (not to say that it didn't before...) so it wasn't too hard to write a script to sync up his iTunes with my music database:


rsync \
-av \
--compress \
--port=[redacted for security :P] \
--stats \
--exclude=Movies \
--exclude=Podcasts \
carter@192.168.1.110:'/media/disk/My\ Music/iTunes/iTunes\ Music/' \
'/Volumes/External Space/Music/Music_from_Max/'


So there -- taking the music from my disk (its only in iTunes so when I boot Vista it syncs up with my iPhone... Vista? iPhone? now I'm embarrassed...) and putting it on his disk. There are a few issues we ran in to. First of all, his username is capitalized on the Mac -- Carter -- and lowercase on my computer. So thats why, even though its almost the same user, we have carter@192... . Also, the port=### option doesn't seem to work on rsync on the Mac. I run ssh on a non-standard port, but even with the --port= option it kept defaulting to 22. So, we edited /etc/ssh/ssh_config and put my port in there. Also, we're doing this with rsa keys so the script could run on a crontab! Then, every time I download music from this millenium, he would get it.

Finally. No more Red Hot Chili Peppers.

rsync

The rsync utility is very cool. Its cool because its fast, effective and the syntax for the command is relatively simple. Its a great way for the uber-paranoid to avoid the Cloud. [Side note: if you are not uber-paranoid and do not mind the Cloud, check out Dropbox <-- shameless referral link. But you get extra space with that link, too! 2 GB -> 2.25 GB]

You can get rsync to do automatic, incremental backups for you, although I'm still working on that part. The best I have so far is to sync 1 or more computers with your server computer (I sync my computers at home using my dad's computer in San Francisco [static IP] as the server computer).

To do it (assuming the server is running an rsync deamon [more on that here], and your server name is in the /etc/hosts file), each computer (except for the server computer) needs 2 identical scripts. Mine are:


rsync \
--verbose \
--archive \
--compress \
--update \
coffee:~max/test1 ~/rsync/


and


rsync \
--verbose \
--archive \
--compress \
--update \
~/rsync/test1 coffee:~max/

This keeps the folder "test1" in my home directory on coffee (the server computer) and my ~/rsync directory on my home computers. The options are:
--verbose = tell me whats going on as its happening!
--archive = this is a cocktail of options: recurse into the directories, copy symlinks as symlinks and preserve permissions.
--compress = use compression to speed up the transmission, but use up more CPU
--update = don't overwrite newer files on the receiver (server) computer

The rest of the syntax is the same as a copy command. If you use a remote computer, preface its directory structure with the name of the computer followed by ":".

Another very very useful option when testing all this is --dry-run. rsync will go through all the steps it would have taken to make the sync without actually transferring any files. Especially useful if you use the --delete flag, which deletes files on the server computer that are no longer present on the source computer. Because of rsync's trailing slash issues, its easy to delete the contents of an entire directory with one wrong "/".

Here's a simple script if you're just using rsync to backup:

#!/bin/sh
BUNAME=$(date +%A)
rsync \
--verbose \
--archive \
--compress \
--update \
--backup \
--backup-dir=~max/$BUNAME/ \
~/rsync/test1/ coffee:~max/test1/


Because of the --backup and --backup-dir=... options, when rsync is going to change a file, it first makes a copy of the file to the directory specified by $BUNAME (which is defined as the day of the week i.e. Thursday) and then overwrites the file in the main directory (test1, in this case). This way, if you accidentally change a file you didn't mean to, you have backups of it. You could go so far as to make hourly directories within the day-of-the-week directories with some simple shell scripting...