emax

Subversion is another awesome piece of software for keeping files in sync across multiple computers. Not quite suited for backing up pictures or anything like that — but hey, its great for big coding projects! Usually with big projects, there are different levels of access. For example, Bob’s algorithm may be super secret and only he and Bill should be able to see the code for it. But, Barry may be logging in to the same SVN server and need access to other code. There’s an app for that. I mean, there’s a feature for that. Also, perhaps not everyone should have access to the server’s file, or even an account on the server. Well, there’s a feature for that, too. Anyway, here’s the procedure/setup I came up with after lots of research/trial-and-(hopefully not catastrophic) error.

Make sure at least YOU can ssh into the server (or better yet, be sitting in front of it and skip the next few steps). Its handy to use an RSA key so you can log in automatically and don’t have to type your password every time you want to execute a subversion command. So:

sh-keygen -t rsa

will generate you a key. Then:

rsync -a ~/.ssh/id_rsa.pub server:~/.ssh/

Before we switch over to the server, if you’re using a non-standard port, in /etc/ssh/ssh_config, add:

Host [server]
Port [port #]
Host * [this line should be there already, it just tells you where to put the 2 preceeding lines]

Next, ssh over to the server and:

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
rm ~/.ssh/id_rsa.pub

The >> instead of just > puts the key at the END of the file instead of overwriting, so if you already have a computer set up in your authorized_keys file, this will preserve that setup.

Now you should be able to log in with no password using ssh [server].

Part 2 coming soon!

Max Sobell backup, code, linux, rsync, software

http://drupal.org/handbook/customization/tutorials/beginners-cookbook

Looks very powerful, easy to use and cool! And best of all, its GPL’d.

http://maxsobell.com/drupal/

http://pingv.com/videos/1/what-drupal-57-seconds

Max Sobell code, nothing_useful, software

For as much as Apple and Google are in bed together, the way the iPhone sets up gmail by default kinda sucks. So here’s a better way (from Google):

http://mail.google.com/support/bin/answer.py?hl=en&answer=77702

Then go to “Settings” – “Mail Contacts and Calendars” – “your-account@gmail.com” – “Advanced” and under “Mailbox Behaviors” change the following:

Drafts Mailbox: On the Server – “Drafts”
Sent Mailbox: On the Server – “Sent Mail”
Deleted Mailbox: On My iPhone – “Trash”

The Deleted Mailbox seems strange, but they recommend it here: http://mail.google.com/support/bin/answer.py?answer=78892# I had it on the gmail server until I decided I better double check before posting online, and it seemed to be working just fine.

Max Sobell iPhone

.flac? .ogg? .mp3? Wouldn’t it be great to just have one standard? That sounds like a tagline, but I’m going nowhere with it.

Very useful instructions to convert .flac to .wav to .mp3.

This guy got it right: mp(3).

Max Sobell nothing_useful

The rsync scipts will be in the next edition of A Practical Guide to Linux Commands, Editors, and Shell Programming! The author and I go way back…

Anyway, its (again) a handy utility. My roommate used to play all 90′s music (and not the good stuff) on his Mac. The rsync utility comes default in Mac OS X (not to say that it didn’t before…) so it wasn’t too hard to write a script to sync up his iTunes with my music database:


rsync \
-av \
--compress \
--port=[redacted for security ] \
--stats \
--exclude=Movies \
--exclude=Podcasts \
carter@192.168.1.110:'/media/disk/My\ Music/iTunes/iTunes\ Music/' \
'/Volumes/External Space/Music/Music_from_Max/'


So there — taking the music from my disk (its only in iTunes so when I boot Vista it syncs up with my iPhone… Vista? iPhone? now I’m embarrassed…) and putting it on his disk. There are a few issues we ran in to. First of all, his username is capitalized on the Mac — Carter — and lowercase on my computer. So thats why, even though its almost the same user, we have carter@192… . Also, the port=### option doesn’t seem to work on rsync on the Mac. I run ssh on a non-standard port, but even with the –port= option it kept defaulting to 22. So, we edited /etc/ssh/ssh_config and put my port in there. Also, we’re doing this with rsa keys so the script could run on a crontab! Then, every time I download music from this millenium, he would get it.

Finally. No more Red Hot Chili Peppers.

Max Sobell backup, linux, rsync, software

I keep hearing people say: “man, I really should back up my computer” with this look of dread on their faces. Its really not so bad. Here are a few ways, ranging from the absolute easiest to the most nerdly-satisfying Oh, and they’re all free (at least for the most basic option)

1. Dropbox: for the novice who needs to back up some documents, a couple pictures maybe, or even some music. All in all, less than 2 GB (“500 songs”, in Apple-speak). 2 GB is free. If you’re willing to pay, you can back up your whole system with Dropbox — 50 GB is $100/yr and 100 GB is $200/yr. The best part about Dropbox is that its got version control (covered in “subversion” later on) so its great for programming projects or group projects — its also got a “share folder” feature so everyone can be working from the same files. ALSO: Dropbox works for Mac, Linux and Windows. Awesome! My favorite new piece of software. The downside (or upside, depending) is that its all got to be in one folder called “Dropbox” on Linux or “My Dropbox” on Windows. I put mine on my desktop for easy access. If you want to just sync your whole desktop or documents across a couple computers (Windows only), try www.sugarsync.com. They now offer (because of competition from Dropbox) a free 2 GB option.

2. Picasa web: Backing up pictures can be very space-consuming. If you use Picasa (now for Mac, Windows and Linux) and have a google account, you can back up to the google servers. They give you 1 GB for free (pictures only) which will hold a lot of pictures at medium resolution. Now, you could just use 1 GB of your Dropbox space, but most cameras now take pictures that are really high resolution (and take up more space on your disk) — Picasa reduces the size of your photos before it uploads them, so its a simple way to make sure you still have a decent quality copy of your pictures even if you disk fails.

These are the 2 most basic options that cover most things. However, some people are concerned with privacy. These sites keep things private (Dropbox is password protected unless you “share” a folder with someone, and Picasa has 2 options to keep things private — “unlisted”, which means the link isn’t public, or “sign in required to view” which means even if you have the link, only people you explicitly share it with can see your pictures).

Some people are even more concerned with privacy and are resolutely “anti-cloud”. Cloud computers are, generally, computers that you can’t see (because you don’t know where they are) that have your data on them. So, webmail (gmail, hotmail, yahoo, etc.) is a cloud-based service. There are some people (you know who you are) who don’t use cloud-based services. This makes life very difficult. So, for these people, there are a few other options. I’ve gotta write about these later because I have to finish an rsync script first

Max Sobell backup, linux, software

Awesome update. There seems to be one issue — syncing contacts. I have them syncing over the web to google contacts. When I upgraded to 3.0, a TON of duplicate contacts appeared, as well as some that I didn’t put in there in the first place (google’s “suggested contacts” — lots of sale-######@craigslist.org, etc). There seem to be 2 workarounds:

1. If you have a mac, create a blank group in mac contacts and choose to sync with that group, as well as over the web w/ google. Somehow that works, but I don’t have a mac, so I didn’t pay much attn.

The one that worked for me was:
2. Choose “overwrite existing contacts” in the Info tab on your iPhone in iTunes. Also, on the iPhone itself, go to Settings – Mail, Contacts and Calendars, and select your account (xxxx@gmail.com), that says “Contacts, Calendars” under it. Turn off Contacts and Calendars, and when the pop-up comes up, choose “Delete.” That will clear your iPhone of some of its contacts. Then, turn off contact syncing from within iTunes. Sync your iPhone. Go back to the menu you were at on your iPhone, and turn back on the contacts and calendars, and when the pop-up comes up, choose “Delete” again. Now, you are writing to a blank iPhone. Phew.

Thanks, http://discussions.apple.com/thread.jspa?threadID=2048666.

Edit: the most useful iPhone/gCal link: m.google.com/sync (from your iPhone).

Max Sobell iPhone

This is the final project for a bioinformatics class I took last semester:

http://err.bio.nyu.edu/courses/index.php/V22.0480_Final_Project

I worked on Project 1. The goal was to find a scalable algorithm to cluster large data sets (something that couldn’t all fit into memory) with arbitrary dimensions. We wrote a SQLite adapter to grab n data points at a time (based on k, the number of expected clusters), and then ran a clustering algorithm on those n points, storing the results in memory (if the data set was large enough to require it, we could write them back to the SQLite DB). After num.iter iterations, we run the algorithm again on the result set to get our final medoids. From there, its relatively easy to assign each point to a medoid, forming the final clusters.

Max Sobell code, dead reckoning, finals, statistics

The rsync utility is very cool. Its cool because its fast, effective and the syntax for the command is relatively simple. Its a great way for the uber-paranoid to avoid the Cloud. [Side note: if you are not uber-paranoid and do not mind the Cloud, check out Dropbox <-- shameless referral link. But you get extra space with that link, too! 2 GB -> 2.25 GB]

You can get rsync to do automatic, incremental backups for you, although I’m still working on that part. The best I have so far is to sync 1 or more computers with your server computer (I sync my computers at home using my dad’s computer in San Francisco [static IP] as the server computer).

To do it (assuming the server is running an rsync deamon [more on that here], and your server name is in the /etc/hosts file), each computer (except for the server computer) needs 2 identical scripts. Mine are:


rsync \
--verbose \
--archive \
--compress \
--update \
coffee:~max/test1 ~/rsync/


and


rsync \
--verbose \
--archive \
--compress \
--update \
~/rsync/test1 coffee:~max/

This keeps the folder “test1″ in my home directory on coffee (the server computer) and my ~/rsync directory on my home computers. The options are:
--verbose = tell me whats going on as its happening!
--archive = this is a cocktail of options: recurse into the directories, copy symlinks as symlinks and preserve permissions.
--compress = use compression to speed up the transmission, but use up more CPU
--update = don’t overwrite newer files on the receiver (server) computer

The rest of the syntax is the same as a copy command. If you use a remote computer, preface its directory structure with the name of the computer followed by “:”.

Another very very useful option when testing all this is --dry-run. rsync will go through all the steps it would have taken to make the sync without actually transferring any files. Especially useful if you use the --delete flag, which deletes files on the server computer that are no longer present on the source computer. Because of rsync’s trailing slash issues, its easy to delete the contents of an entire directory with one wrong “/”.

Here’s a simple script if you’re just using rsync to backup:

#!/bin/sh
BUNAME=$(date +%A)
rsync \
--verbose \
--archive \
--compress \
--update \
--backup \
--backup-dir=~max/$BUNAME/ \
~/rsync/test1/ coffee:~max/test1/


Because of the --backup and --backup-dir=... options, when rsync is going to change a file, it first makes a copy of the file to the directory specified by $BUNAME (which is defined as the day of the week i.e. Thursday) and then overwrites the file in the main directory (test1, in this case). This way, if you accidentally change a file you didn’t mean to, you have backups of it. You could go so far as to make hourly directories within the day-of-the-week directories with some simple shell scripting…

Max Sobell backup, code, linux, rsync, software

Well, finals are almost over, but mid-week there was a Computer Science Showcase in the Courant building at NYU. Our Robotics class had a corner where the Rovios were playing soccer and the 3πs were following lines. My Rovio, someone else’s 3π:

Adjusting the Rovio to run on the hardwood floors with the different lights was a challenge — I had to change the coefficients for distance to ball and goal and the threshold for detecting the ball. The floor made it particularly challenging because the reflection of the lights is about the same color as the tennis ball (to the robot) and so sometimes it thinks the ball is somewhere its not. Add in human interference, as a bunch of us were jammed into a 9′x9′ area, and it was chaos. Also, I added a victory dance at the end, thanks to a request in the comments.

This is someone’s 3π — I have my code for this, but its not as fast as this one. Wow. The 3π uses 5 infrared sensors on the front end, along with some C++ and a PID controller to follow the line.

Also, the professor just sent out an email; looks like our class was picked up here:
http://www.slashgear.com/wowwee-rovio-taught-to-play-soccer-0642907/
and here:
http://www.robocommunity.com/blog/entry/15953/Rovio-Learns-Soccer-Bends-it-
Like-Beckham/

Rovios face off.

Blurry 3π racer

Max Sobell 3pi, Lisp, Rovio, code, finals, line follow, lush, robot