Roy's Techblog

I've always felt that the ubiquitous windowed model of computing with a single pointing device is inefficient. It seems that the folks at 10/GUI agree, and so does Lukas Mathis and others like Scott at ISO50

I'm excited by R. Clayton Miller's 10/GUI, primarily because it is a step in the right direction. Reducing the reliance on haphazardly placed windows and the introduction of a single multi-purpose and very powerful input device are great ideas. For a long time, I've been thinking on and off about how to get out of the windowed paradigm. I've never succeeded.

I dislike the mouse - its clumsy, inaccurate, and hurts. I like keyboard controls, but my fingers get confused beyond two meta keys at a time. I love my multitouch touchpad on my MacBook. I also love the multitouch on the iPhone, but find its screen frustratingly small and the ergonomics for lengthy use to be poor. The obstruction problem mentioned in the 10/GUI video constantly occurs.

All that said, I don't feel that the ideas presented in 10/GUI address power-users, software developers, or multi-document/multi-monitor use very well. I don't even think it deals with normal business multi-tasking very well either.

Frequently, a business user will require two or three applications at a time:

• Communication tools (possibly two) such as MSN/iChat/GTalk/Skype, etc.
• Email
• Reference materials (documents, browsers, help files, emails(s), customer service database screens, administrative tools, etc.)
• Working task (eg. a new document, a new email, administrative tools, etc.)

I believe that a better solution would be to be able to break the workspace into zones and affix "windows" to these zones. Zones could be associated with portions of each display.

I believe that this type of arrangement would help users effectively use increasingly common large and wide-screen monitors, while still performing tasks that require reference materials.

This arrangement should scale well to software developers and power users with dual (or more) monitors, too:

Things that might exist in the Optional Dashboard Widgets would be weather, stocks, news/rss headlines, email alerts, calendar previews, parcel tracking.

Things that might be in the Communications Tool(s) section include VOIP, Instant Messaging, Email status, RSS feeds, Microblogging, etc.

My feeling is that by being semi-windowed in a structured manner, people will be able to continue to use their computers in a more familiar way. This should ease the transition to a new window management philosophy while remaining somewhat familiar. This would allow users to continue to perform their tasks in a familiar way in a novel GUI, thus easing transition and likelihood of success.

Twice now, I’ve helped friends recover lost photos from flash cards. Both friends are technical people who generally know what they’re doing … but they both got bitten by bugs in commercial software that left them without their photos for one reason or another.

The loss of photos is a very upsetting event to a photographer. Fortunately, if you stop using the card the instant you discover the problem, more than likely, your photos can be recovered. If you never use the on-camera delete to remove anything but the most recently taken photo(s), you’re in even better shape, since no fragmentation will have happened!

So how do you recover photos when there’s NO filesystem information left? Turns out its pretty easy for some file formats and tools that ignore trailing junk in files. The approach is simple: snapshot the raw card contents and then look for the start-of-file signature for your specific type of photo files— but most importantly, look for it aligned at the start of 512-byte boundaries —the typical “block” size.

To determine your file’s start signature, you’ll need a few sample files from your camera. If you have a Canon Rebel XTi and shoot in RAW mode, you’ll be getting CR2 files. These files have a nice long signature at the start, making detection and recovery a breeze.

To determine if there’s a good unique signature, you can do something like this:

for f in IMG*.CR2; do hexdump -n 16 $f; done | sort | uniq -c
     6 0000000 4949 002a 0010 0000 5243 0002 3006 0001
     6 0000010

If there were 6 files in the directory, and you only get 2 lines of output, you’ve found yourself a reliable 16-byte signature. More than enough to detect the start of files in most cases, especially when aligned to the start of a 512-byte block.

The above signature is what’s needed for a CR2 file.

To obtain the necessary image of the flash card (I don’t recommend ever working directly on the flash card when doing recovery—so we read it once and save it for future processing).

So how do you grab the contents of the flash for safe processing? Under Linux, FreeBSD and OSX (and other unix platforms), you use dd. We do this because we don’t want any extra headers… just the raw bytes. This ensures the disk remains aligned at 512-byte boundaries. Some disk image containers might happen to keep things aligned to the block boundaries. I’ve never checked.

The specific instructions for OSX are as follows:

1. Using a card reader, mount the flash card (like usual: just insert it)
2. Start Terminal.app and run df to find the device name of the newly mounted flash card. We’re specifically interested in the /dev/diskNsN device name, since we’re going to need to directly access it. Here’s an example:

   /dev/disk4s1      999344    978464     20880    98%    /Volumes/EOS_DIGITAL

3. Next, we need to unmount the disk without causing the device to be removed. We can either use unmount /dev/disk4s1 or go to Disk Utility, select the right volume, then use the “Unmount” toolbar icon to unmount it without ejecting it.
4. Finally, we create the image we’re going to work with using dd.

   dd if=/dev/disk4s1 of=flashimage.dat

   Depending on the size, speed of the card, your card reader, and USB interface, this could take a long time. If you need to know how far it’s gotten, open a new Terminal window and run “du -h flashimage.dat”

Once the copy of the image has been made, we’ll want to run a quick recovery script. This script relies on the fact that Canon Raw conversion utilities tend to ignore trailing junk. If yours don’t, grab http://cybercom.net/~dcoffin/dcraw/:”dcraw” (available via http://www.macports.org/:”Mac Ports”) and convert the files to a format you can use (like TIFF).

Here’s the recovery script I hacked together to recover missing CR2 files for my friend:

#!/usr/bin/perl

use strict;
use warnings;

my @signature = qw(49 49 2a 00 10 00 00 00 43 52 02 00 06 30 01 00);
my $signature = pack("H*", join("", @signature));
my $siglen = length($signature);
my $blocksize = 512;

open(IN, "flashimage.dat") || die "$!";

# Skip 2gb to get past existing files
seek(IN, 2 * 1024 * 1024 * 1024, 0);

my $block = "";
my $imgno = 0;
while (read(IN, $block, $blocksize)) {
    if (substr($block, 0, $siglen) eq $signature) {
        print "starting $imgno\n";
        open(OUT, sprintf(">found%04d.CR2", ++$imgno));
    }
    print OUT $block unless !$imgno;
}

The end result should be a whole bunch of .CR2 files named from found0001.CR2 through the final number.

PS – The above should work for JPEG files, but JPEG headers aren’t as big/consistent. The above technique has been proven to work for a Pentax *ist with a fully erase card (unerase couldn’t be performed) before any files were written and a Canon Rebel XTi where 1/2 the card had been filled back up with new photos. Some files might be corrupt due to fragmentation. Basically, what I’m saying is: the only two cameras I know of so far that write their files in a sane manner without fragmentation (unless holes are created by erasing files on the camera) are the above two cameras.

Picking a good bag seems to be hard to do. It’s possible I’m just fussy, but I do have some particular needs.

An ideal bag must carry:

1. a 15” or so laptop
2. a textbook or novel
3. a moleskine notebook (or two)
4. miscellaneous papers like bills, printed academic papers, a magazine, or printouts needed for working at home
5. a couple CDs or DVDs (in paper covers)
6. a few pens/pencils in an external pouch
7. some business cards
8. a power adapter
9. various cables (usb, serial, ipod adapters)
10. full sized headphones when travelling
11. an iPod 5G
12. a camera the size of a Canon G7

And it must:

1. be rain resistant
2. be padded
3. be comfortable to carry when fully loaded

And finally:

1. I’d like it not to squash my papers.

So far, the candidates that have potential bags are:

1. Crumpler
2. Tom Bihn
3. Timbuk2
4. BOOQ
5. and I had been pondering Courierware
6. and Manhattan Portage

One of the problems I have had is deciding whether I want a backpack or shoulder-bag of some form (such as a messenger bag). You’d think this would be an easy choice, but it turns out not to be.

I currently carry a Brenthaven Pro 15/17 for work, but it belongs to work, and I’m changing jobs. This bag is great, but its not quite ideal. The laptop compartment takes up a very large percent of this bag, and while the design lets you overstuff it, I find I quite often damage papers or books. There really isn’t that much space to carry papers around, although the divided paper compartment is a great thing. The top pocket on the front of it for pencils and other little things is wonderful, and the top-loading zipped up slot for the laptop is great, but I always worry that it won’t hold out in a downpour. The bottom cable compartment is also great, and the front flap with space for your miscellaneous stuff is great… But its just not quite ideal for me because things get crumpled too easily.

A friend carries around a 2-3 year old Tom Bihn Brain Bag and loves it, but I’m not sure if it fully addresses what I need. It certainly has some nice pockets and the Freudian Slip looks like it’ll help avoid crumpled papers. I can’t remember if it had a top pocket that I’ve grown to love on the Brenthaven.

I went and had a look at a Timbuk2 Classic Messenger at a local shop in both the Large and Medium sizes, to get an idea for how the bags would be in their Laptop Messenger line. The Large is frickin’ huge—perhaps too big. I’d probably damage everythig in my bag all the time. It just doesn’t have enough organizational aids. The Medium looks much better size-wise, and had a nice amount of open space… But I think I’d need both the laptop compartment and a divider. I’ve also looked at people’s photos of what they carry in their Commute at Flickr to see if I think it’d do. Sadly, I think its just a little too small.

Maybe the Super Ego is what I need?

I’ll post again when I finally make up my mind.

Update: Since originally writing this article, a friend suggested I look at Chrome as well.

Update: I finally picked the Tom Bihn Brain Bag brain bag with a Freudian Slip and a Snake Charmer (for when traveling) and an appropriately sized Horizontal Brain Cell which holds the laptop snugly and with plenty of padding. I’ve taken to using the Brain Cell to haul the laptop around the house. I’ll post a review of my bag soon.

Using a really awesome general purpose proxy called Delegate, its really easy to proxy and encrypt NNTP for news clients that don’t use SSL:

For example, to proxy from the local machine (say a Mac), just run:

sudo ./delegated ADMIN=admin@admin.invalid STLS=fsv -P119 SERVER=tcprelay://news.giganews.com:563

Have you ever been plagued by slow performance in MySQL? This video/talk made by MySQL to Google covers many really useful tips!

A 45-minute must-listen if you use MySQL.

One of the things that has bugged me for ages is that /etc is a combination of system files and local configuration files…

I don’t think it should be both!

At lunch today, a colleague and I discussed the idea of making /etc system-only files, and union mounting it with a directory to hold locally modified files.

I’m going to have to try this to see if it works. If it does, then perhaps its time to adopt a new way to manage /etc for FreeBSD.

It turns out that this works, and very well! The only trick is upgrading. And removing files… I’ll have to try using below-mounts and relocating the real /etc to a different directory, and having /etc be the local modifications but with the system /etc/accessible.

If you’re like me, you have several macs. Probably one at work, and one at home… or a desktop and a laptop… And somehow you also have an iPod.

One of the great things about an iPod is you can use it to carry your music with you between two computers (of the same platform)... The trick is convincing iTunes that this is okay.

I managed to convince iTunes on my work laptop that my iPod was okay to use. To do this, I did, roughly, the following:

• Turned off automatic synchronization on my source computer (the one that syncs to the iPod – my desktop)
• Quit iTunes on the laptop.
• Overwrote ~/Library/iTunes on the laptop with the one from my desktop. (this wasn’t absolutely necessary)
• Copied over ~/Library/Preferences/com.apple.iTunes.eq.plist ~/Library/Preferences/com.apple.iTunes.plist and ~/Library/Preferences/com.apple.iTunesHelper.plist
• Copied over ~/Music/iTunes Library and ~/Music/iTunes Music Library.xml from my desktop.
• Started iTunes on the laptop.
• Removed my playlists and deleted all songs—they didn’t really exist anyhow, since I had nothing in ~/Music on the laptop.
• Plugged in the iPod.
• Enabled “Manually manage music and videos”, “Enable disk use” and then on each of the tabs (Music, Movies, TV Shows, Podcasts, Photos, Contacts, Games), unckeched the “Sync” checkbox at the top.

I can now select my iPod’s playlists and songs to listen to them instead of having a copy of the library on both computers.

I decided some time back to try to keep a daily techblog. While chatting with some friends online, it occurred to me that I really should keep one. After all, it shouldn’t be too hard to keep a browser window open and add stuff to a blog post throughout the work day, chronicling my daily adventures in tehchspace. I’m sure other people run into situations that I do from time to time.

Today started with a horrendous mess in the subversion repository: development, production, and the subversion repository all didn’t match. Thankfully the following handy command(s) were all I really needed to do most of the work:

diff -qr -x .svn production development
diff -qr -x .svn production subversion

I love diff -qr for rapid determination of equivalence of directory structures.

And now for a bitchfest about the default colour schemes for ls and vim for RedHat linux. They suck! They make me turn off colourization as fast as I can. To do so easily for ls, “touch .dircolors” in your homedir.

I’m not sure if its an age thing, but none of the people here at work my age or older like colorls. They see that it could be useful, but find the colour scheme painfully difficult to use, especially the blues on blacks. Almost all of us seem to have black background terminals, or at least dark blue. I don’t see anyone using white terminal backgrounds except one guy, but he doesn’t like colorls either.. Being that I love colorised editors, the dislike of colorls must be related to the chocie of colours. A friend totally agrees.

The default colors are: exfxcxdxbxegedabagacad. I find this painful.

This is a scheme I just game up with:

LSCOLORS='BxdxfxfxcxfxfxCxCxbxbx'

• Directories – bright red, Directories writeable by others – dark red.
• Symlinks – brown/yellow (depending on system colors)
• Special files (sockets, pipes, block, character) – magenta.
• Executables – green.
• SetUid/SetGID executables – bright green.

The net result is a touch of colour on your default background. This color string is tuned for black terminals, however.

Furthermore, GNU LS doesn’t do the same thing as BSD ls, and instead you want a .dircolors file:

NORMAL 00       # global default
FILE 00         # normal file
DIR 31;1          # directory
LINK 33         # symbolic link.
FIFO 35         # pipe
SOCK 35         # socket
DOOR 35         # door
BLK 35          # block device driver
CHR 35          # character device driver
ORPHAN 31     # symlink to nonexistent file
EXEC 32          # This is for files with execute permission:

Note the lack of suffix highlighting.

Today, I ran across an in-browser Javascript Shell

It works great.