South's frozen models are big liars

March 07, 2013 at 02:53 PM | Python, Django | View Comments

Consider South's orm object, http://south.readthedocs.org/en/latest/ormfreezing.html, which provides "frozen-in-time" versions of ORM models:

(Pdb++) from my_app.models import MyModel
(Pdb++) MyModel
<class 'my_app.models.MyModel'>
(Pdb++) orm['my_app.MyModel']
<class 'my_app.models.MyModel'>
(Pdb++) orm['my_app.MyModel'] == MyModel
False

Notice that the magic South frozen model appears to be in the same namespace as my real model: my_app.models!

This is at best stupid, and at worst potentially dangerous.

It makes perfect sense that South needs some mechanism for providing "frozen models" (ie, which match the database schema at the time of migration, of the current schema), but they are emphatically not the same as the real models (don't have any of the real methods, etc), so it should be made very clear that they are different! There are many ways this could be done, but one very simple step would be putting them in a different namespace, which makes it clear that these models are frozen, and references the migrations they belong to:

(Pdb++) orm['my_app.MyModel']
<class 'south.frozen_models.my_app.models_0003.MyModel'>

Additionally, this is a bad code smell which could potentially lead to bugs: because the "frozen model class" is lying about where it's from, any code which relies on the module path will do surprising things. In fact, it's such a big problem that pickle explicitly checks for this case:

(Pdb++) import pickle
(Pdb++) pickle.dumps(orm['my_app.MyModel'])
*** PicklingError: Can't pickle <class 'my_app.models.MyModel'>:
    it's not the same object as my_app.models.MyModel
Permalink + Comments

Tips for Managing a Django Project

June 22, 2011 at 04:44 PM | Python, Django | View Comments

During the time I've spent with Django, I've picked up a couple tricks for making life a little bit less terrible.

First, split the project project into three (or more) parts, each with its own settings.py: the main project, the development environment and the production environment. For example, my current project, code named eos, has a directory structure something like this:

eos/
    .hg/
    .hgignore
    manage.py
    run
    eos/
        __init__.py
        settings.py
        templates/
        urls.py
        ...
    hacking/
        __init__.py
        settings.py
        db.sqlite3
        ...
    production/
        __init__.py
        settings.py
        run.wsgi
    ...

The eos/ directory is more or less a standard Django project (ie, created by django-admin.py startproject), except that eos/settings.py does not have any environment-specific information in it (for example, it doesn't have any database settings or URLs).

The hacking/ and production/ directories also contain settings.py files, except they define only environment specific settings. For example, hacking/settings.py looks a bit like this:

from eos.settings import *
path = lambda *parts: os.path.join(os.path.dirname(__file__), *parts)

DATABASE_ENGINE = "sqlite3"
DATABASE_NAME = path("db.sqlite3")

DEBUG = True

While production/settings.py contains:

from eos.settings import *

DATABASE_ENGINE = "psycopg2"
DATABASE_NAME = "eos"
DATABASE_USER = "eos"
DATABASE_PASSWORD = "secret"

DEBUG = False

Then, instead of configuring Django (ie, calling setup_environment) on eos.settings, it is called on either hacking.settings or production.settings. For example, manage.py contains:

...
import hacking.settings
execute_manager(hacking.settings)

And production/run.wsgi contains:

...
os.environ["DJANGO_SETTINGS_MODULE"] = "production.settings"
...

Second, every settings.py file should contain the path lambda:

path = lambda *parts: os.path.join(os.path.dirname(__file__), *parts)

It will make specifying paths relative to the settings.py file very easy, and completely do away with relative-path-related issues. For example:

MEDIA_ROOT = path("media/")
DATA_ROOT = path("data/")
DATABASE_NAME = path("db.sqlite3")

Third, there should be scripts for running, saving and re-building the environment. I use two scripts for this: run and dump_dev_data. By default the run script calls ./manage.py runserver 8631 (specifying a port is useful so that web browsers can distinguish between different applications - keeping passwords, history, etc. separate). Run can also be passed a reset argument, which will delete the development database and rebuild it from the dev fixtures. These fixtures are created by the dump_dev_data script, which calls ./manage.py dumpdata for each application, saving the data to fixtures named dev (these fixtures are committed along side the code, so all developers can work off the same data).

So, for example, when I'm developing a new model, my workflow will look something like this:

... add new model to models.py ...
$ ./run reset # Reset the database adding the new model
... use the website to create data for the new model ...
$ ./dump_dev_data # Dump the newly created data
$ hg commit -m "Adding new model + test data"
Permalink + Comments

Django settings.py mini-tip: the `path` lambda

January 17, 2010 at 10:26 PM | Python, Django | View Comments

There are a few things I do every time I start a new Django site, and one of those things is add my path lambda to the top of settings.py:

    import os
    ROOT = os.path.abspath(os.path.dirname(__file__))
    path = lambda *args: os.path.join(ROOT, *args)

From then on, it's drop-dead simple to create absolute paths which are relative to the root of the project (or, at least, the directory which contains settings.py). For example:

    >>> path('media/')
    '/Users/wolever/code/review_anywhere/media/

And some other places where it is useful:

    DATABASE_ENGINE = 'sqlite3'
    DATABASE_NAME = path('db.sqlite3')
    ...
    TEMPLATE_DIRS = (
        path('templates/'),
    )
    ...
    DATA_DIR = path('data/')
    ...
Permalink + Comments

Overriding global URLs in Django Tests

October 09, 2009 at 02:38 AM | Django | View Comments

One thing that has bugged me about testing Django apps is that I couldn't figure out how to override the global URL settings, which meant that app tests were tightly coupled with the global URL scheme.

For example, to test the adder app, I used Client calls like this:

resp = Client().get("/toys/calculator/adder/1+2")

Which were, of course, totally lame because they a tight coupling between the adder app and the project it lived in.

But there is a better (albeit poorly documented) way: overwriting django.conf.settings.ROOT_URLCONF!

There are two ways to do it: hand-rolling a solution, or using TestCase.urls.

Here is a quick example of both. First, using TestCase.urls:

import adder

class AdderTests(TestCase):
    urls = adder.urls

    def test_adder(self):
        resp = Client().get("/1+2")

And then, the hand-rolled solution I've been using with django-nose:

_old_root_urlconf = [ None ]
def setup_module():
    _old_root_urlconf[0] = settings.ROOT_URLCONF
    settings.ROOT_URLCONF = urls

def teardown_module():
    settings.ROOT_URLCONF = _old_root_urlconf[0]

def test_adder():
    resp = Client().get("/1+2")
Permalink + Comments