Most of the time, my little hacks are command line apps, without much flash. But lately I find myself doing more and more artsy side-projects.

Aside from my one at work (which is pretty cool!), I’ve been working on writing a Getting Things Done webapp here at home. I named it Sencha after my favorite green tea at Samovar, and the time when I feel most in charge of things.

After designing and implementing the data model, I’m now spending some time on the interface. I sketched this on the BART to work a few days ago. Tonight I finally got to a stage where I’m seeing some resemblance!

It’s just weird that it’s a role I’d never thought I’d like playing or have any talent for. It’s also not something they teach you in engineering school (though admittedly, I never took 160 or 169, but am not sure what level of detail they go into)

And, I mean, me? With a sketchbook?

I’ve been toying with the idea for the past few days since I got inspired by the MapReduce chapter in Beautiful Code. Mostly it’s a cute little project that’s trendy and scratches an itch of mine — namely, how do I utilize the full power of my little cluster of machines?

My end goal is to bootstrap this on top of the raytracer Lin and I did in CS184 as sort of a proof-of-concept, and run speed tests. Other simple ideas include MP3 encoding (split the wav file, give each machine a piece, and then mp3wrap them together at the end) or if I get really daring, video re-encoding (which would be damn cool for saving off my HD streams)

The idea on top of all of this is that: you split up a problem, you do something to the pieces (Map), you can emit intermediate values (to be further sorted or combined in Reduce), and then you can combine the problem back into the original spec in some way (or not, depends on how you write Reduce). So long as there’s a common NAS (Granted, it’s no GFS, but then, I’m not dealing with petabytes) there can be the necessary file output sharing. SSHFS counts too — it’s just slower.

It’s also cute in that it can spread other time-intensive tasks across machines. It doesn’t even have to transfer files if what you want to compute is somehow representable. Since I’m writing it all in Python (or C# where necessary) a good example is to spread minimax subtrees across machines to make a faster, smarter CS188 Pacman (which would merely return the value of the root-node move — all across TCP).

The backend interface looks something like this right now:

[02:37] michener@enjolras:~$ telnet 192.168.0.16 6278
Trying 192.168.0.16...
Connected to 192.168.0.16.
Escape character is '^]'.
Welcome to the PumaCreed Server on fantine.
Type 'help' for details.
> help
help jobs ls newjob stat quit shutdown
> stat
Computer Name   System  Ranking Threads Description
fantine         Linux   9000    1       2.6.21-2-686 #1 SMP Wed Jul 11 03:53:02 
enjolras        Darwin  9000    1       9.1.0 Darwin Kernel Version 9.1.0: Wed O
> shutdown
Connection closed by foreign host.
[02:38] michener@enjolras:~$

Not much, but it’s a start. The machines know about each other, there’s networking and threads and config files going on (praise be unto Twisted) — not to mention the start of a MapReduceProgram class from which all code run on the cluster should inherit (Or at least implement the interface of).

We’ll see how it goes.

My Ubuntu install on eponine had been freezing. Just up and freezing. So I built a FreeBSD install from scratch — decided that, for the desktop, it was lame (though the next time I overhaul fantine, my server, it’s going FreeBSD; what I learned makes it kinda rule for non-X11 things). So I tried 64-bit Ubuntu — which had the same problem, but fortunately worked just as well as 32-bit in terms of proprietary functionality (nVidia drivers, Flash, etc). Currently sitting on 64-bit Fedora 8; but it’s not apt to stay too long.

(Mostly, I’m coming to suspect it’s the ACPI — I’ve tried the nVidia beta driver to no avail)

Meanwhile….

I’m downgrading to XP on cosette.

For all intents and purposes, I did so about a week and a half ago — just booted my old XP partition. Hey, sudden burst of comfort, speed, and it worked right. Huzzah. But then I came to remember why I tried Vista in the first place — this old partition is starting to feel “dirty”. I need a reinstall.

The reason I hate reinstalling Windows, of course, is installing the software I like. If ever there were a major plus to Linux, it’s package management; go out, grab your favorite software, all with one command and be up and running just-like-that. I was able to switch distros like mad recently and it shows. You just can’t do that in Windows.

Until I went by the CSUA office today and was told of a magical XP image, with all the greatest hits installed. I grabbed it to my iPod and am now munging my partition tables but good (ntfsresize -P --force --force /dev/sda2) to throw the image over my Vista install, killing it forever. It’ll run overnight. And if it doesn’t work, I’ve got my trustier XP CD sitting here.

Meanwhile, I’m pondering putting a fresh install of Leopard on marius, my old Mac mini.

It’s the time of year to clean house, digitally.

And in case you’re wondering and like my naming scheme of Les Miserables characters; I’m writing this on enjolras, my MacBook Pro.

That’s what was wrong with my Leopard install. I was using (an uncompatible) hack to run Witch, a really nice power tool (which I hope, once they update, will remain Leopard-compatible)

My take on Leopard, now that it works:

• New Finder really beats the pants off the old one. Not a hard task, but an important one.
• You can almost feel the underlying API changes (I recompiled MacVim from SVN because of this)
• It’s a little more shiny. Stupid blue folders are gone, though I’m not a huge fan of the Tango-esque ones.
• Stacks are genuinely useful.
• They got Spaces wrong. I mean, this one should have been hard to mess up, given the precedent, but they did. It’s a fine virtual desktop manager (a feature in the X11 world for AGES) but it could use a few more keybindings — it’s great that I can switch desktops logically, but why must I use the mouse to move applications around? (Needs a Move-Current-App-in-Direction key combo)
• Also, closing a program will switch Spaces on you. This is wrong. It should work it’s way down the apps in the current Space, not bounce all over based on last-usage. arg.
• Time Machine is about as useful as Dashboard — which is to say, oh, it’s cute, but it’s disabled and apt to stay that way.

So you may wonder “What’s the point?” if I just kinda shot down the big features.

It’s for developers.

They’ve added official Ruby/Cocoa bindings. Python TK apps no longer require X11 (as I found out from running Pacman). Both of these are at the current version and include many nifty non-default libraries (Rails, Twisted, NumPy, etc, etc…). Not to mention all the optional dev tools (I recompiled the latest MacVim, as stated earlier. This is UNIX thinking). I’m thinking that interpreted-languages-as-second-class-GUI-programs is coming to an end. Ruby is the new Java. Make the devs happy, and they will develop for your platform.

MS is doing it too. Visual Studio Express was no accident. And today I hear about F# — which is essentially Microsoft OCaml (as C# is Microsoft Java)

The real power of Leopard — as with the real power of Vista — is yet to be seen. It comes down the line, about a year and a half from now, when the freelance dev masses come along. Vista arguably added a number of genuinely useful API stuff — the .NET 3.0 backings come as an obvious example. Microsoft, logically, is backing it’s own horse. Apple knows the flaws of writing in straight Objective-C and so opens up to more casual devs and UNIX devs by backing open, interpreted languages.

In fact, I bet they added PyObjC by default, instead of giving a user guide to it…

Python 2.5.1 (r251:54863, Oct  5 2007, 21:08:09)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import objc
>>>

Oh look….

So I got Leopard today, and, throwing caution to the wind, decided to upgrade my system.

bzzt Wrong! I am hosed!

So I’m lifting my userdata out onto my iPod — praise be unto the gods of the command line — and am going to do a fresh install.

I guess it all works out; but I would be lying if I weren’t disappointed (so far)

However, playing with the display computers at TSW I discovered that python is ver. 2.5 and has Twisted and NumPy by default. Which is awesome.

Network card works better (maybe when it got the updates something clicked) but the network is SLOWER. I dunno what they did wrong, but my Samba NAS takes forever, until I read this: http://jamespo.org.uk/wp/archives/190 — This seems to make it work fairly well

Still best to get video drivers straight from nVidia. Sorry MS.

I like the new Explorer. It has it’s faults, but it’s an improvement. Sidebar, meanwhile, is stupid and nobody uses it.

Still no major incompatibilities. So far so good.

Just finishing up these few quick notes before I write something completely different.

• UAC is annoying, but thank god you can turn it off (along with other Windows nags, which I appreciate)
• Network fails on boot — but request an IP and it’s good again. Step up from XP, which didn’t have a driver for my NIC by default, but step down in reliability. Maybe fixed if I get a new (non-MS) driver.
• Why do I need to do a full Windows Update to get a (very common) Sound Blaster Audigy driver? You think it’d work out of the box…
• Why can’t I turn off hiding hidden files and keep my desktop clean from desktop.ini files at the same time? XP did this…
• More reboots to get things set up than you can shake a stick at — but this is a failing of Windows in general
• So far, no incompatibility. So far.
• C:\Users! It’s like /home, but not! This is an improvement over C:\Documents and Settings.
• Whatever happened to the Run.. command?

Ubuntu worked out of the box with all this. It got my nVidia driver right. It got the network right. It got sound right. And it comes with a shitload of software. I’m not saying that’s perfect either — it doesn’t like my huge display until i edit xorg.conf, for example. And 36 updates is a light update cycle (in beta, at least) And individual apps break sometimes. But by and large, the fastest OS to install to get to baseline functionality these days is Linux, and that’s not at all the way it used to be.

Microsoft did whiff at Vista. Perhaps not as badly as they could have, but it’s nowhere near what a 6 year dev cycle should produce. Vista would have been about right if it was released with a little less graphical glitz but a similar feature set in 2003/2004.

By the by, I’m not saying Apple is a big winner either. Leopard comes out this month, so we’ll see what happens — but there they go for vertical integration.

More as I figure it out.

In my news cycle this week was this article on Improving your photography through classical art. I took the time to try it out today (and by “time”, I mean “five minutes”) and applied the first colorful art piece I could think of offhand, Seurat’s A Sunday Afternoon on the Island of La Grande Jatte to an appropriate swing photo with similar colors. I find the theme of the pieces similar.

This is the result, “A Saturday Afternoon Dancing on the Plaza of Sproul”:

Pretty cool, huh?

So today I woke up and rewrote my crappy parser from scratch, having had a showertime epiphany about not only how to write a good one, but write one with features we’ll need for the next part of the project.

In the end, I came out with exact same linecount. So I was +0 in lines but plus several million in functionality.

We didn’t get it in just on time because our grammar-rewriter was, well, mangled. This bothered me a little somewhere deep inside. Until I had two quick beers and took another shower. Upon emerging I had an inkling about what to do. Drunken? Perhaps. But that sort of drunken clarity that makes things possible.

So I scrapped darch’s rewriter and started over. At 3 in the morning. I wrote and wrote and things started to just fall into place. A slight change of order here, a minor off-by-one hack there, one piece slyly using some nice output from another piece… and in 2.5 hours I had it up and running. Which is better than busted. And much shorter, cleaner, and compartmentalized than it previously was.

My line count for the day: -70 My contribution: Enough to make me feel like I deserve my trip to Seattle for Spring Break

Right now I’ve got so much hack energy that I can’t sleep if I wanted to. Oh, I’ll crash real soon now, but this is the first time in a while I’ve felt the rush. And it’s this that makes me think that everything’s going to be okay once I leave school and my life is somehow on track — which I was worried about earlier (eating at the dorms makes me feel old).