Tips for Managing a Django Project

June 22, 2011 at 04:44 PM | Python, Django | View Comments

During the time I've spent with Django, I've picked up a couple tricks for making life a little bit less terrible.

First, split the project project into three (or more) parts, each with its own settings.py: the main project, the development environment and the production environment. For example, my current project, code named eos, has a directory structure something like this:

eos/
    .hg/
    .hgignore
    manage.py
    run
    eos/
        __init__.py
        settings.py
        templates/
        urls.py
        ...
    hacking/
        __init__.py
        settings.py
        db.sqlite3
        ...
    production/
        __init__.py
        settings.py
        run.wsgi
    ...

The eos/ directory is more or less a standard Django project (ie, created by django-admin.py startproject), except that eos/settings.py does not have any environment-specific information in it (for example, it doesn't have any database settings or URLs).

The hacking/ and production/ directories also contain settings.py files, except they define only environment specific settings. For example, hacking/settings.py looks a bit like this:

from eos.settings import *
path = lambda *parts: os.path.join(os.path.dirname(__file__), *parts)

DATABASE_ENGINE = "sqlite3"
DATABASE_NAME = path("db.sqlite3")

DEBUG = True

While production/settings.py contains:

from eos.settings import *

DATABASE_ENGINE = "psycopg2"
DATABASE_NAME = "eos"
DATABASE_USER = "eos"
DATABASE_PASSWORD = "secret"

DEBUG = False

Then, instead of configuring Django (ie, calling setup_environment) on eos.settings, it is called on either hacking.settings or production.settings. For example, manage.py contains:

...
import hacking.settings
execute_manager(hacking.settings)

And production/run.wsgi contains:

...
os.environ["DJANGO_SETTINGS_MODULE"] = "production.settings"
...

Second, every settings.py file should contain the path lambda:

path = lambda *parts: os.path.join(os.path.dirname(__file__), *parts)

It will make specifying paths relative to the settings.py file very easy, and completely do away with relative-path-related issues. For example:

MEDIA_ROOT = path("media/")
DATA_ROOT = path("data/")
DATABASE_NAME = path("db.sqlite3")

Third, there should be scripts for running, saving and re-building the environment. I use two scripts for this: run and dump_dev_data. By default the run script calls ./manage.py runserver 8631 (specifying a port is useful so that web browsers can distinguish between different applications - keeping passwords, history, etc. separate). Run can also be passed a reset argument, which will delete the development database and rebuild it from the dev fixtures. These fixtures are created by the dump_dev_data script, which calls ./manage.py dumpdata for each application, saving the data to fixtures named dev (these fixtures are committed along side the code, so all developers can work off the same data).

So, for example, when I'm developing a new model, my workflow will look something like this:

... add new model to models.py ...
$ ./run reset # Reset the database adding the new model
... use the website to create data for the new model ...
$ ./dump_dev_data # Dump the newly created data
$ hg commit -m "Adding new model + test data"
Permalink + Comments

Python security tip: urllib/urllib2 will read `file://` URLs

June 05, 2011 at 11:03 PM | Python | View Comments

I discovered, entirely by accident, that urllib2.urlopen, urllib.urlretrieve, and probably others, will happily read file:// urls and filesystem paths. For example:

>>> import urllib, urllib2
>>> urllib.urlretrieve("database_connection_settings.txt", "/tmp/temp_file")
('/tmp/temp_file', <mimetools.Message instance at 0x…>)
>>> urllib2.urlopen("file:///dev/urandom").read(10)
'\xf1r?\x0fC\x86p\x05\xa4\xdd'

This means that applications which blindly urlopen untrusted URLs (for example, from RSS feeds) are potentially vulnerable to information disclosure and denial of service attacks.

Permalink + Comments

Scroll Wheel Emulation for the TrackMan Marble

June 04, 2011 at 04:26 PM | Uncategorized | View Comments

I recently had to get a new mouse, and because Logitec has discontinued my mouse of choice, the TrackMan Wheel[0], I got a TrackMan Marble.

Unlike every other modern mouse, the Marble doesn't have any sort of scroll wheel… But fortunately this can be emulated in software. I've done this on my Mac using KeyRemap4MacBook and a custom private.xml. I've also been told that Marble Mouse Scroll Wheel (mirrored marbleinst.exe) works well on Windows.

[0]: It has been replaced by, of all things, a wireless version. Why does a stationary mouse need to be wireless? I've got no idea. Maybe it's a conspiracy between Logitec and Duracell?

Permalink + Comments

Four Kinds of Pair Programming

June 02, 2011 at 12:16 AM | Uncategorized | View Comments

In thinking about the pair programming that I've done, I realized that I can separate all of my past partners (all 6-8 of them) into four categories:

  • Equally skilled
  • Less skilled
  • Much less skilled
  • Skilled in a different domain

Of these, I have found the “less skilled” partners to be the most frustrating. Not because of their personality[0], but because during the sessions I've felt like there is a mutual understanding of the goal, but I would just be able to get there more quickly (for example, because I'm more experienced with the libraries or environment). I haven't felt this way with the “much less skilled” partners because I believe there is a mutual understanding that the session more akin to a tutorial… So, in my mind, the goal becomes “teach my partner” instead of “solve a problem”.

The times I've paired with an “equally skilled” partner do not stand out in my memory, so I assume they went okay.

My favourite pairing experience was when a friend and I had to figure out the inner workings of some example C code for a micro controller. My friend knew a lot about micro controllers, I knew a lot about C, and together we were able to reverse engineer the hideous example code much more quickly than either of us alone could have. It was a lot of fun.

[0]: as I write this, I don't have any specific people in mind, just memories of my experience.

Permalink + Comments