The Cave

MySQL Project

noreply@blogger.com (Max) — Sat, 24 Oct 2009 05:44:00 +0000

I've started a MySQL wiki for a chapter of A Practical Guide to Red Hat Linux. I'm learning as I write it up, so feel free to correct or change anything! Here's the link 

Karmic Koala!

noreply@blogger.com (Max) — Tue, 20 Oct 2009 06:04:00 +0000

Programming Contest

noreply@blogger.com (Max) — Mon, 19 Oct 2009 15:01:00 +0000

This year's problems for the ICPC:
http://www.acmgnyr.org/year2009/problems.shtml

SVN Part 2

noreply@blogger.com (Max) — Thu, 08 Oct 2009 19:43:00 +0000

Now that we can log in automatically (SVN Part 1), we can do some cool stuff. This summer I managed a SVN server for a company that has multiple programs, each of which had different developers working on them. Each of these programs is stored as a repository in SVN. For example, developer A needs access to repository 1 and 2 and developer B needs access to repository 1 and 3, etc. To make it even trickier, in repository 1, developer A should have access to the entire repository, while developer B should only have access to the directory devB in repository 1. Also, every time a developer makes a change to a file, the change needs to be logged with their username.

This is where the RSA keys come in. Each repository should be owned by a different user. For example, /usr/bin/svnserver/repository1 is owned by user repo1, /usr/bin/svnserver/repository2 by repo2, etc. Put any developer who needs access to [repo]'s RSA pubkey in the /home/[repo]/.ssh/authorized_keys file, where [repo] is the name of the repository's owner (repo1, repo2). Configure authorized_keys like:

command="/usr/bin/svnserve -t -r /var/svn/ --tunnel-user=[developer's username]",no-port-forwarding,no-pty,no-agent-forwarding,no-X11-forwarding ssh-rsa [developer's RSA pubkey]== [developer's username]

This enables developers to check out [repository] using: svn+ssh://[repo]@[server]/[repository]. While we are sending the information over SSH, the developer does not have a SSH account and can do nothing but use SVN to check out (svn co) and update (svn ci) repositories owned by a user (repo1, repo2) whose authorized_keys file contains the developer's pubkey. The last column in the authorized_keys file is the comment line. SVN automatically uses this line as the comment when a developer commits code to the SVN repository.

The last part, giving only partial access to a repository, coming in Part 3.

SVN Part 1

noreply@blogger.com (Max) — Mon, 03 Aug 2009 14:52:00 +0000

Subversion is another awesome piece of software for keeping files in sync across multiple computers. Not quite suited for backing up pictures or anything like that -- but hey, its great for big coding projects! Usually with big projects, there are different levels of access. For example, Bob's algorithm may be super secret and only he and Bill should be able to see the code for it. But, Barry may be logging in to the same SVN server and need access to other code. There's an app for that. I mean, there's a feature for that. Also, perhaps not everyone should have access to the server's file, or even an account on the server. Well, there's a feature for that, too. Anyway, here's the procedure/setup I came up with after lots of research/trial-and-(hopefully not catastrophic) error.

Make sure at least YOU can ssh into the server (or better yet, be sitting in front of it and skip the next few steps). Its handy to use an RSA key so you can log in automatically and don't have to type your password every time you want to execute a subversion command. So:

sh-keygen -t rsa

will generate you a key. Then:

rsync -a ~/.ssh/id_rsa.pub server:~/.ssh/

Before we switch over to the server, if you're using a non-standard port, in /etc/ssh/ssh_config, add:

Host [server]
Port [port #]
Host *

[this line should be there already, it just tells you where to put the 2 preceeding lines]

Next, ssh over to the server and:

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
rm ~/.ssh/id_rsa.pub

The >> instead of just > puts the key at the END of the file instead of overwriting, so if you already have a computer set up in your authorized_keys file, this will preserve that setup.

Now you should be able to log in with no password using ssh [server].

Part 2 coming soon! :P

Testing my feed to Drupal

noreply@blogger.com (Max) — Tue, 28 Jul 2009 16:54:00 +0000

http://drupal.org/handbook/customization/tutorials/beginners-cookbook

Looks very powerful, easy to use and cool! And best of all, its GPL'd.

http://maxsobell.com/drupal/

http://pingv.com/videos/1/what-drupal-57-seconds

Gmail on iPhone

noreply@blogger.com (Max) — Wed, 22 Jul 2009 17:02:00 +0000

For as much as Apple and Google are in bed together, the way the iPhone sets up gmail by default kinda sucks. So here's a better way (from Google):

http://mail.google.com/support/bin/answer.py?hl=en&answer=77702

Then go to "Settings" - "Mail Contacts and Calendars" - "your-account@gmail.com" - "Advanced" and under "Mailbox Behaviors" change the following:

Drafts Mailbox: On the Server - "Drafts"
Sent Mailbox: On the Server - "Sent Mail"
Deleted Mailbox: On My iPhone - "Trash"

The Deleted Mailbox seems strange, but they recommend it here: http://mail.google.com/support/bin/answer.py?answer=78892# I had it on the gmail server until I decided I better double check before posting online, and it seemed to be working just fine.

Can't we all just get along?

noreply@blogger.com (Max) — Sat, 18 Jul 2009 21:17:00 +0000

.flac? .ogg? .mp3? Wouldn't it be great to just have one standard? That sounds like a tagline, but I'm going nowhere with it.

Very useful instructions to convert .flac to .wav to .mp3.

This guy got it right: mp(3).

APGT Linux Commands, Editors, and Shell Programming

noreply@blogger.com (Max) — Fri, 17 Jul 2009 23:08:00 +0000

The rsync scipts will be in the next edition of A Practical Guide to Linux Commands, Editors, and Shell Programming! The author and I go way back... ;)

Anyway, its (again) a handy utility. My roommate used to play all 90's music (and not the good stuff) on his Mac. The rsync utility comes default in Mac OS X (not to say that it didn't before...) so it wasn't too hard to write a script to sync up his iTunes with my music database:


rsync \
-av \
--compress \
--port=[redacted for security :P] \
--stats \
--exclude=Movies \
--exclude=Podcasts \
carter@192.168.1.110:'/media/disk/My\ Music/iTunes/iTunes\ Music/' \
'/Volumes/External Space/Music/Music_from_Max/'

So there -- taking the music from my disk (its only in iTunes so when I boot Vista it syncs up with my iPhone... Vista? iPhone? now I'm embarrassed...) and putting it on his disk. There are a few issues we ran in to. First of all, his username is capitalized on the Mac -- Carter -- and lowercase on my computer. So thats why, even though its almost the same user, we have carter@192... . Also, the port=### option doesn't seem to work on rsync on the Mac. I run ssh on a non-standard port, but even with the --port= option it kept defaulting to 22. So, we edited /etc/ssh/ssh_config and put my port in there. Also, we're doing this with rsa keys so the script could run on a crontab! Then, every time I download music from this millenium, he would get it.

Finally. No more Red Hot Chili Peppers.

Backup software

noreply@blogger.com (Max) — Sat, 11 Jul 2009 05:37:00 +0000

I keep hearing people say: "man, I really should back up my computer" with this look of dread on their faces. Its really not so bad. Here are a few ways, ranging from the absolute easiest to the most nerdly-satisfying :) Oh, and they're all free (at least for the most basic option)

1. Dropbox: for the novice who needs to back up some documents, a couple pictures maybe, or even some music. All in all, less than 2 GB ("500 songs", in Apple-speak). 2 GB is free. If you're willing to pay, you can back up your whole system with Dropbox -- 50 GB is $100/yr and 100 GB is $200/yr. The best part about Dropbox is that its got version control (covered in "subversion" later on) so its great for programming projects or group projects -- its also got a "share folder" feature so everyone can be working from the same files. ALSO: Dropbox works for Mac, Linux and Windows. Awesome! My favorite new piece of software. The downside (or upside, depending) is that its all got to be in one folder called "Dropbox" on Linux or "My Dropbox" on Windows. I put mine on my desktop for easy access. If you want to just sync your whole desktop or documents across a couple computers (Windows only), try www.sugarsync.com. They now offer (because of competition from Dropbox) a free 2 GB option.

2. Picasa web: Backing up pictures can be very space-consuming. If you use Picasa (now for Mac, Windows and Linux) and have a google account, you can back up to the google servers. They give you 1 GB for free (pictures only) which will hold a lot of pictures at medium resolution. Now, you could just use 1 GB of your Dropbox space, but most cameras now take pictures that are really high resolution (and take up more space on your disk) -- Picasa reduces the size of your photos before it uploads them, so its a simple way to make sure you still have a decent quality copy of your pictures even if you disk fails.

These are the 2 most basic options that cover most things. However, some people are concerned with privacy. These sites keep things private (Dropbox is password protected unless you "share" a folder with someone, and Picasa has 2 options to keep things private -- "unlisted", which means the link isn't public, or "sign in required to view" which means even if you have the link, only people you explicitly share it with can see your pictures).

Some people are even more concerned with privacy and are resolutely "anti-cloud". Cloud computers are, generally, computers that you can't see (because you don't know where they are) that have your data on them. So, webmail (gmail, hotmail, yahoo, etc.) is a cloud-based service. There are some people (you know who you are) who don't use cloud-based services. This makes life very difficult. So, for these people, there are a few other options. I've gotta write about these later because I have to finish an rsync script first ;)

iPhone OS 3.0

noreply@blogger.com (Max) — Mon, 22 Jun 2009 17:02:00 +0000

Awesome update. There seems to be one issue -- syncing contacts. I have them syncing over the web to google contacts. When I upgraded to 3.0, a TON of duplicate contacts appeared, as well as some that I didn't put in there in the first place (google's "suggested contacts" -- lots of sale-######@craigslist.org, etc). There seem to be 2 workarounds:

1. If you have a mac, create a blank group in mac contacts and choose to sync with that group, as well as over the web w/ google. Somehow that works, but I don't have a mac, so I didn't pay much attn.

The one that worked for me was:
2. Choose "overwrite existing contacts" in the Info tab on your iPhone in iTunes. Also, on the iPhone itself, go to Settings - Mail, Contacts and Calendars, and select your account (xxxx@gmail.com), that says "Contacts, Calendars" under it. Turn off Contacts and Calendars, and when the pop-up comes up, choose "Delete." That will clear your iPhone of some of its contacts. Then, turn off contact syncing from within iTunes. Sync your iPhone. Go back to the menu you were at on your iPhone, and turn back on the contacts and calendars, and when the pop-up comes up, choose "Delete" again. Now, you are writing to a blank iPhone. Phew.

Thanks, http://discussions.apple.com/thread.jspa?threadID=2048666.

Edit: the most useful iPhone/gCal link: m.google.com/sync (from your iPhone).

Kmeds Algo

noreply@blogger.com (Max) — Sun, 14 Jun 2009 03:26:00 +0000

This is the final project for a bioinformatics class I took last semester:

http://err.bio.nyu.edu/courses/index.php/V22.0480_Final_Project

I worked on Project 1. The goal was to find a scalable algorithm to cluster large data sets (something that couldn't all fit into memory) with arbitrary dimensions. We wrote a SQLite adapter to grab n data points at a time (based on k, the number of expected clusters), and then ran a clustering algorithm on those n points, storing the results in memory (if the data set was large enough to require it, we could write them back to the SQLite DB). After num.iter iterations, we run the algorithm again on the result set to get our final medoids. From there, its relatively easy to assign each point to a medoid, forming the final clusters.

rsync

noreply@blogger.com (Max) — Thu, 11 Jun 2009 18:19:00 +0000

The rsync utility is very cool. Its cool because its fast, effective and the syntax for the command is relatively simple. Its a great way for the uber-paranoid to avoid the Cloud. [Side note: if you are not uber-paranoid and do not mind the Cloud, check out Dropbox <-- shameless referral link. But you get extra space with that link, too! 2 GB -> 2.25 GB]

You can get rsync to do automatic, incremental backups for you, although I'm still working on that part. The best I have so far is to sync 1 or more computers with your server computer (I sync my computers at home using my dad's computer in San Francisco [static IP] as the server computer).

To do it (assuming the server is running an rsync deamon [more on that here], and your server name is in the /etc/hosts file), each computer (except for the server computer) needs 2 identical scripts. Mine are:


rsync \
--verbose \
--archive \
--compress \
--update \
coffee:~max/test1 ~/rsync/

and


rsync \
--verbose \
--archive \
--compress \
--update \
~/rsync/test1 coffee:~max/

This keeps the folder "test1" in my home directory on coffee (the server computer) and my ~/rsync directory on my home computers. The options are:
--verbose = tell me whats going on as its happening!
--archive = this is a cocktail of options: recurse into the directories, copy symlinks as symlinks and preserve permissions.
--compress = use compression to speed up the transmission, but use up more CPU
--update = don't overwrite newer files on the receiver (server) computer

The rest of the syntax is the same as a copy command. If you use a remote computer, preface its directory structure with the name of the computer followed by ":".

Another very very useful option when testing all this is --dry-run. rsync will go through all the steps it would have taken to make the sync without actually transferring any files. Especially useful if you use the --delete flag, which deletes files on the server computer that are no longer present on the source computer. Because of rsync's trailing slash issues, its easy to delete the contents of an entire directory with one wrong "/".

Here's a simple script if you're just using rsync to backup:


#!/bin/sh
BUNAME=$(date +%A)
rsync \
--verbose \
--archive \
--compress \
--update \
--backup \
--backup-dir=~max/$BUNAME/ \
~/rsync/test1/ coffee:~max/test1/

Because of the --backup and --backup-dir=... options, when rsync is going to change a file, it first makes a copy of the file to the directory specified by $BUNAME (which is defined as the day of the week i.e. Thursday) and then overwrites the file in the main directory (test1, in this case). This way, if you accidentally change a file you didn't mean to, you have backups of it. You could go so far as to make hourly directories within the day-of-the-week directories with some simple shell scripting...

Computer Science Showcase

noreply@blogger.com (Max) — Fri, 08 May 2009 05:08:00 +0000

Well, finals are almost over, but mid-week there was a Computer Science Showcase in the Courant building at NYU. Our Robotics class had a corner where the Rovios were playing soccer and the 3πs were following lines. My Rovio, someone else's 3π:

Adjusting the Rovio to run on the hardwood floors with the different lights was a challenge -- I had to change the coefficients for distance to ball and goal and the threshold for detecting the ball. The floor made it particularly challenging because the reflection of the lights is about the same color as the tennis ball (to the robot) and so sometimes it thinks the ball is somewhere its not. Add in human interference, as a bunch of us were jammed into a 9'x9' area, and it was chaos. Also, I added a victory dance at the end, thanks to a request in the comments.

This is someone's 3π -- I have my code for this, but its not as fast as this one. Wow. The 3π uses 5 infrared sensors on the front end, along with some C++ and a PID controller to follow the line.

Also, the professor just sent out an email; looks like our class was picked up here:
http://www.slashgear.com/wowwee-rovio-taught-to-play-soccer-0642907/
and here:
http://www.robocommunity.com/blog/entry/15953/Rovio-Learns-Soccer-Bends-it-
Like-Beckham/

Rovios face off.

Blurry 3π racer

Rovio works!

noreply@blogger.com (Max) — Tue, 05 May 2009 13:30:00 +0000

The point of the whole project is to get the ball into the goal.

The algorithm goes:

take a picture
convert it from RGB to YUV
mask out the ball and the posts into 2 separate matrices
find the ball
turn to it
rotate around the ball until you find the goal
shoot!

And if it were only 7 lines of code my life would have been much easier for the last little while ;) Here's all the code. And here's the Rovio in action:

Woot!

The next step is to clean up and compile the code to get it ready to play soccer against the other Rovios!

Class Rovios

noreply@blogger.com (Max) — Mon, 04 May 2009 05:59:00 +0000

http://www.cs.nyu.edu/~yann/2009s-V22-0480-001/media.html

The professor put up some pics/videos of the robots in action, err... more in circles.

Picasa Facial Recognition

noreply@blogger.com (Max) — Mon, 04 May 2009 05:48:00 +0000

The facial recognition feature on Picasa (online only) is _amazing_. This was back when I had "tagged" < 1000 pics. Now I'm at ~4k and it is right about 75% of the time. This screenshot is from when I was having it search for pictures of my dad; it pulled up pictures of me and thought they could either be me, my dad, or my brother. Wow.

There is one minor issue with this one though. It was doing pretty well (except for that 4th one... don't know who that is) until the very last pic. Hm..... maybe its the shades?

Finals Snack

noreply@blogger.com (Max) — Sat, 02 May 2009 01:35:00 +0000

I know we're not quite to finals week yet, but this is the pre-final's cram week that weakens everyone; then finals come in for the final blow, no pun intended. So here's a quick snack from TJ's: Quinoa, some cilantro/chicken pot stickers (err I guess you call them dumplings over here. Btw I waited _in_ line for them, and I'm putting this _on_ line), an egg (duh) and corn + garlic fried in sesame oil and then some spinach thrown in at the last minute. And Fairway olive oil over the top ;)

Back to the 3pi

noreply@blogger.com (Max) — Wed, 22 Apr 2009 04:47:00 +0000

It follows a line... stops... turns around... and goes back to where it started! Finally...

The code is here.

That outside line is for its next trick ;)

The Robot is getting smarter...

noreply@blogger.com (Max) — Sat, 18 Apr 2009 16:26:00 +0000

Now it can find goal posts! And its supposed to be able to shoot the ball into the goal... that is, after it finds the ball and the posts and lines up properly, which is proving to be tricky...

We convert the RGB picture it takes into YUV, and then analyze that, so it filters the color of the tennis ball better. Then, we use this awesome time-saving library called CCAnalysis to find the middle of the posts and the ball, we turn until the ball is inbetween the posts, and then... we shoot!!

(Lisp) is mind-numbing

noreply@blogger.com (Max) — Tue, 31 Mar 2009 02:03:00 +0000

Well, not entirely Lisp... its actually (lush). But the parentheses... Oy. Thank god for Emacs paren highlighting. Here's the whole thing: robotest.lsh, but here's just a little sample of the beauty of (lush) syntax:

(setq len (* h (/ (- (* k f) y) (+ (* k y) f))))

Just like http://xkcd.com/224/ and http://xkcd.com/297/!

The whole point of this was to take in an image (top), mask out the color (bottom) by first eroding, to eliminate noise, and then dilating, to get back the origninal big blob of color (the eraser):

Then that formula (above) tells the robot how far the object is from it, based on how many pixels down the screen the middle of it is, how high off the ground the robot's lens is, and its focal point (-371). That distance formula took a whole classroom a while to figure out....

This is for a Rovio robot (which is controlled over wifi), and that picture was taken from the robot.

Robots attack in subtle ways...

noreply@blogger.com (Max) — Thu, 19 Mar 2009 15:52:00 +0000

Trying very hard to get this little schmuck to follow a line (done.) and then go back to where it started (not even close...).

I'm still figuring out how to work this blog site, and I think they have a tool to do this, but its very useful to use

mogrify -resize 640 *.jpg

and, if you need the package,

sudo aptitude install imagemagick

Brunch in NYC

noreply@blogger.com (Max) — Sun, 15 Mar 2009 14:23:00 +0000

Why is finding a place to have brunch in the city so hard? Well, its because there are about 5 places every block serving brunch, and its impossible to find menus for most of them online. So, will my brunch come with a free drink and some coffee? Or just some eggs and toast? Hm... my current favorite is Blue Water Grill, which is ridiculously expensive the other 6 days of the week, but Sunday Brunch is a bargain. You can get an entree for $13, with drink, coffee, and snooty atmosphere gratuit.

1st problem with blogger

noreply@blogger.com (Max) — Tue, 03 Mar 2009 16:44:00 +0000

It won't let me put my birthday as 2/29! It says its an "invalid date"...

hey

noreply@blogger.com (Max) — Tue, 03 Mar 2009 16:26:00 +0000

So maybe I'll start weblogging about what I do 10 hours/day.... play with computers! Google thinks weblogging is spelled wrong, and suggests "we blogging, backlogging and flogging" but recognizes its abbreviation -- blogging!