Entries by David Wolever

Ok guys, it's pretty simple: if you're working in a Turing-complete environment and you've got paths which start with a '/', you're doing it wrong. Very wrong!

Todays offender: WSGI scripts.

Here's the first line of a common WSGI script:

app_path = '/path/to/application'

See a problem there? That's right, the absolute path!

How could this be solved? Simple: put the WSGI script in /path/to/application, then get the app_path using: app_path=os.path.abspath(os.path.dirname(__file__)). Problem solved.

</rant>

A couple of days ago, I complained about FlexUnit's Theories. Well, with a bit of encouragement from @drewbourne, I broke down and wrote a proper parameterized testrunner:

    [RunWith("utils.testrunners.ParameterizedRunner")]
    class AdditionTests {
        public static var numbersToTest:Array = [
            [1, 2, 3],
            [4, 5, 9],
            [-1, 1, 0]
        };

        [Parameterized("numbersToTest")]
        public function testAddition(a:int, b:int, expected:int):void {
            assertEqual(a+b, expected);
        }
    }

And that code will do exactly what you'd expect: run three test cases, one for each of the inputs. If one test fails, the others will still run. Helpful error messages will be provided when a test fails.

The source can be downloaded from: http://gist.github.com/299871 (and it will be getting a new home when ever I get around to releasing all of my AS utilities)

Look useful? Give it a try and tell me what you think – I'd love to know.

tl;dr: learn what Alpha can do, add a keyword search for it.

Remember Wolfram|Alpha? Wolfram's attempt at a computer knowledge engine that got a little bit of hype, then drifted into obscurity?

Well, it has been making me pretty happy recently, and maybe it can make you happy too. Here's my trick:

First, learn what it can and can't do: it won't help much with a broken xorg.conf… But if you need to convert units (especially units of storage), get information about a date, compare stock information or do anything with a mathematical equation, Alpha is incredible.

(yes, there is nothing there that's unique to Alpha… But that's not the point. The point is that, once you know what it can do, you can, on the first try, get the information you need. There's no looking through Google search results (even Google's calculator takes a couple of tries (compare: alpha: 4 CAD in USD).)

Second, make it really, really easy to use: add a keyword search (if you're in Safari, use Stand) so searching Alpha is just as easy as searching the rest of the Internet:

Then just start using it

I've just been playing around with FlexUnit 4's nifty new "Test Theories"… And I have come to the conclusion that, right now, they are basically worthless.

Here's why:

• They aren't documented. At all. There are approximately two examples on the internet, and the wiki page is a joke. Nothing explains what algorithm is used to calculate which DataPoints are applied to which theories.
• They make test error messages less helpful. If a a theory fails, instead of getting a helpful error message like "Theory `checkUrl("this is a bad url")` failed with message: could not determine protocol", it gives a message like this: "checkUrl urls[1]". Yea, thanks guys.
• They don't do anything "cool". At all. It would be cool if, given five data points, five distinct tests would be generated. It would be cool if one data point could fail while others succeed. Heck, anything which would make them better than a for loop would be cool. But, alas…

So, don't waste your time on Theories just yet. For now, just use a for loop:

[Test]
public function runTestsOnData():void {
    for each (datum in testData)
        doSomeTest(datum);
}

Or, if you want to test the cartesian product of a set of data, use my handy cartesian product function:

[Test]
public function runTestsOnCartesianProduct():void {
    for each (data in cartesian_product(testData0, testData1))
        doSomeTest.apply(this, data);
}

Followup: as Alan (see comments) said, what I really want is a parameterized testrunner. So I've gone ahead and written one. See my post on a Parameterized Testrunner for FlexUnit.

One of my big beefs with DVCSs is their version numbers.

For example, take two changesets I committed today: 6899 and 02a9. Which one is more recent? How many changesets separate them? Without access to a repository, there's no way to tell… But that sort of information can be useful to have.

Two of the three DVCSs I have experience with, bzr and hg, both take steps towards solving this problem.

bzr tries "really hard" to give everything a sequential ID, and uses those IDs in the UI (as far as I can tell) all the time (it does use unique hashes under the hood, but they aren't shown very much):

$ bzr log
------------------------------------------------------------
revno: 3
message:
  A merge
    ------------------------------------------------------------
    revno: 1.1.1
    message:
      A conflicting change

hg assigns local aliases for each changeset, and displays those aliases along with hashes:

$ hg log -r 4:6
changeset:   4:658109dca65b
description:
A merge

changeset:   5:bd8053bf02f1
description:
A conflicting change

And, of course, git doesn't stand for this sort of frivolity and shows pure, unadulterated, hashes:

$ git log
commit aa55884d693c92da6dc96eb7a45c9ecd774fefc2

    A merge

commit 8e47937468071ae29d385b76ff925d231c65b97b

    A conflicting change

I'd like to see this taken a step further, though: I'd like the changeset hashes themselves to encode some basic information about where they live in the repository.

For example, one way to do this could be using the first two bytes of the hash to store the distance from the root*, and the next two bytes to store a "repository id", which is generated once, when a repository is first cloned or initialized.

So, for example, one of these changeset hashes might look like this: 0afc53bf02f1, and committing again to the same repository would produce 0bfc109dca65.

Of course, this scheme doesn't guarantee anything - it's entirely possible to generate two completely different changesets with the exact same four-byte prefix… But in the general case, this sort of scheme could make it significantly easier to figure out how arbitrary changesets relate to one another.

*: Doug Philips suggested this - thanks.

Be very, very wary of anyone who suggests that "self documenting" code can replace comments.

Case in point: I consider myself a mediocre programmer, who is usually capable of producing mostly coherent code.

Below is a hunk of the code I wrote today - keywords are blue, variables are white, comments are green:

I don't think I could take much away from that without doing a serious disservice to the poor person (probably me) who will be tasked with fixing the inevitable bugs in the above code.

There are a few things I do every time I start a new Django site, and one of those things is add my path lambda to the top of settings.py:

    import os
    ROOT = os.path.abspath(os.path.dirname(__file__))
    path = lambda *args: os.path.join(ROOT, *args)

From then on, it's drop-dead simple to create absolute paths which are relative to the root of the project (or, at least, the directory which contains settings.py). For example:

    >>> path('media/')
    '/Users/wolever/code/review_anywhere/media/

And some other places where it is useful:

    DATABASE_ENGINE = 'sqlite3'
    DATABASE_NAME = path('db.sqlite3')
    ...
    TEMPLATE_DIRS = (
        path('templates/'),
    )
    ...
    DATA_DIR = path('data/')
    ...

(this is a post intended for Google - sorry for bothering you, faithful RSS readers)

I've been setting up encrypted backups for my Linux machine, found that cryptsetup gives a really cryptic and unhelpful error message:

$ cryptsetup luksFormat /dev/sda3 ~/.backupKey
WARNING!
========
This will overwrite data on /dev/sda3 irrevocably.

Are you sure? (Type uppercase yes): YES
Failed to setup dm-crypt key mapping.
Check kernel for support for the aes-cbc-essiv:sha256 cipher spec and verify
that /dev/sda3 contains at least 133 sectors.
Failed to write to key storage.
Command failed.
$

Google will suggest lots of things… But none of them solved my problem.

What was I doing wrong? I needed to run cryptsetup as root.

D'oh.

I picked up an Arduino yesterday, and I was quite dismayed by how much work it took to build/flash from the command line.

After some hours of Googling, though, I finally found an up-to-date Makefile, and it didn't take too much to make it "just work" with the default install of Arduino.app on the Mac.

To use this Makefile, simply drop it into the project sketch directory, then run make then make upload. You may need to tweak some variables (eg, it assumes Arduino.app is located at /Applications/Arduino.app), but that is fairly straight forward.

Get the Makefile from: http://gist.github.com/273821.

Just a note: I haven't had much time to tinker with this system yet, so it may have some serious issues... If so, drop me a comment below.

Also, leave a comment if you want me to let you know if (when?) I update this.

git isn't a backup system.

Neither is Mercurial, Bazaar, Subversion or even (even) CVS CSV.

Version control systems, with the possible exception of SourceSafe, are great at keeping track of code. Why is that? Because they were designed to keep track of code.

Unfortunately, though, the features of a good VCS are entirely different – and often exactly the opposite – of the features which make a good backup system.

Take, for example, file ownership. A good VCS will, very rightly, ignore file ownership: when I check out someone else's code, I should be the owner of those file - not whatever uid originally created them. A good backup system, on the other hand, will do everything in its power to preserve file ownership: when I restore from my backups, I want /etc/shaddow to be owned by root and /home/wolever/ to be owned by wolever.

And ownership is just one example - permissions†, creation and modification times, empty directories‡, hardlinks, xattrs, resource forks, … the list of details that a backup system must keep track of goes on and on.

In fact, there are so many things a backup system can get wrong, there is a project called Backup Bouncer, designed specifically to verify that backup scripts correctly copy all the various bits of metadata tracked by the filesystem.

So, please: if you value your bytes, use a real backup system, not git.

†: Most VCSs only track the 'x' bit - for backup purposes, all bits, including suid bits, must be tracked.
‡: fun fact - Mercurial and git don't track empty directories, but Bazaar does.

(alternate title: The "Fail Early, Fail Often" Principle in Action)

I would like to award Adobe the "best ever implementation of indexOf". Ready for it? Here it is:

override flash_proxy function callProperty(name:*, ... rest):* {
    return null;
}

This beauty can be found in mx.collections.ListCollectionView.

And the kicker? The function's documentation:

/**
 *  @private
 *  Any methods that can't be found on this class shouldn't be called,
 *  so return null
 */

The author clearly knew that unknown methods* shouldn't be called… But instead of doing something sensible – like throwing an exception – they do something wholly nonsensical and return null. Or, of course, they could also have left the function unimplemented, which would result in a strange and unhelpful exception… But, of course, I would expect nothing but the best from the Flex standard library.

Next up: why implicit conversions from null to int are always wrong.

</rant>

*: the magic callProperty method is called on subclasses of Proxy when a method can't be found. For example if bar is not a method of foo, executing foo.bar()will call foo.callProperty("bar").

PS: Instead of calling indexOf, I should have been calling getItemIndex.

In my last post, I showed off some normal-mode data from vim-logging. In this (and the next few) posts, I'll go though my most-used commands and describe how I use them.

(Don't use Vim? This post won't be too interesting… Although you may pick up something useful)

g

My logs contain more than 60,000 references to the 'g' command (eight times more than ':', the next most frequently used command). This may seem surprising at first, but stick with me and let me explain: the 'g' command is something of a 'gateway' command - a prefix for a number of common actions which don't have their own single-character command (for example, gd, goto definition and g?, rot13 encode).

Here is a breakdown of the 'g' suffixes I often use:

• gj and gk: cursor to next row, cursor to previous row ("make j and k do the right thing"). The j and k commands are defined as "move to next line" and "move to previous line", not "move to next screen row" and "move to previous screen row". This is an important distinction to make when long lines are wrapped, and I find "move to next/previous screen row" is a more reasonable default. Protip: use noremap j gj and noremap k gk in your vimrc.

• gg: cursor to top of file. After using gg (for example, to edit some import statements or add a shebang line), I often use '' (tick-tick) to jump back to the line I was editing at the time (for example, ggofrom foo import bar<esc>''). Also, I often use ggVG (go to top, enter line-visual-mode, go to bottom) to select the entire file. From there, I can pipe it to something (for example, :!sort), format it (gw) or copy it to the clipboard.

• gt and gT: go tab - move to next/previous tab.

• gf: goto file - edit the file under the cursor. For example, if the cursor is over the 'e' in import "../eventDispatercherImpl.as", the equivalent of :e ../eventDispatercherImpl.as will be executed.

• gw: go wrap. Wrap the selected text (that is, text selected in visual mode). Try selecting a long line (pressing V), then using gw to wrap it.

That's all the 'g' commands I can think of for now... The next version of vim-logging will log the sub-command used, so I will be able to provide better data in the future.

:

The colon command is fairly obvious: enter command-line mode to run commands like :w or :help :.

I'll write more about this later, when I analyze my most-used ex commands.

d

Ah, d - delete - the programmers best friend.

As with g, I do not have detailed information about how I use d, but I'll list the first few combinations that come to mind when I move my left index finger over the d key:

• dd to delete whole lines
• di( to delete everything inside a pair of parenthesise
• dfx to delete everything between the current position and the next occurrence of 'x'
• dj and dk to delete this and the next/previous line
• d$ to delete from the current position to the end of the line
• d in visual mode to delete the selected text.

Remember when I blogged about vim-logging? Well, I've finally gotten around to interpreting the results.

First, I will cover what I believe to be the most important thing to know about vim: the normal mode commands.

Below is a chart of my 39 (why 39? I can't remember - when I created the graph, that's the number I chose) most-frequently-used normal-mode commands:

I'm sure you didn't notice, but I'll tell you about it anyway: the server that runs wolever.net and codekills.net, lazarus, died of unknown causes two weeks ago.

It took two weeks to get everything fixed because it lives in Toronto and I live in Waterloo… And the first weekend I went back to pick it up… I forgot to pick it up.

Right now the data has been migrated to another machine (the one you're talking to right now), and it is doubtful if lazarus will ever live again.

This is a quickie, but hopefully it'll help someone out there.

I had to setup my laptop so two people could use it to talk on Skype: one audio input (the call), two outputs.

First, I installed Soundflower.

Second, I opened up "Audio MIDI Setup" (/Applications/Utilities/Audio MIDI Setup.app) and created an Aggregate Device (Audio -> Open Aggregate Device Editor). I checked all the output devices, and included the Soundflower device:

Third, I set the system's input and output to Soundflower:

Fourth, I opened up AU Lab (/Developer/Applications/Audio/AU Lab.app) and created a new… thing… with two outputs and one input. After hitting OK the right number of times and getting the main window up, I hit Window -> Show Studio, then selected the Aggregate device on the left. Finally, I checked both the "1" and the "2" on the left side of the "input" to send its output to both output 1 and 2. Some tinkering with the input and output streams (in the Studio) was also necessary.

Finally, in Skype, I set the audio input to the built in microphone (which was fine for me), and everything was peachy!