Why aren't composable test helpers a thing?

November 12, 2013 at 02:20 PM | Python | View Comments

Often test cases require a few stateful "things" to happen during each run, which are often configured in the setUp and tearDown methods of the test case.

For example, some common "things" in applications I've worked on are:

  • Capturing email messages sent during the test case
  • Capturing log messages emitted during the test case
  • Mocking Redis connections
  • Mocking the application's job queue

And the implementation often looks something like this:

class MyRegularTestCase(TestCase):
    def setup(self):
        self.mail = setup_mock_mail()
        self.logs = setup_mock_logger()
        self.redis = setup_mock_redis()
        self.jobs = setup_mock_job_queue()

    def teardown(self):
        self.mail.teardown()
        self.logs.teardown()
        self.redis.teardown()
        self.jobs.teardown()

    def test_foo(self):
        foo()
        self.mail.assert_message_sent("hello, world")
        self.logs.assert_logged("hello, world")
        self.redis.assert_list_contains("foo", "bar")
        self.jobs.assert_job_queued("some_job")

And that is a best case example; most of the time test code doesn't use high-level mocking objects, and setup code is copy+pasted between test classes.

This makes me wonder: why aren't composable test helpers a thing?

For example, a log capturing test helper might look something like this:

class LogCapture(object):
    def __init__(self, logger_name=''):
        self.logger = logging.getLogger(logger_name)

    def setup(self):
        self.records = []
        self.logger.addHandler(self)

    def teardown(self):
        self.logger.removeHandler(self)

    def emit(self, record):
        self.records.append(record)

    def assert_logged(self, message):
        for record in self.records:
            if message in record.getMessage():
                return
        raise AssertionError("No log message containing %r was emitted" %(message, ))

And this sort of composable helper could be used with a TestCase like this:

class MyBetterTestCase(ComposableTestCase):
    mail = MailCapture()
    logs = LogCapture()
    redis = MockRedis()
    jobs = MockJobQueue()

    def test_foo(self):
        foo()
        self.mail.assert_message_sent("hello, world")
        self.logs.assert_logged("hello, world")
        self.redis.assert_list_contains("foo", "bar")
        self.jobs.assert_job_queued("some_job")

I'll be building this kind of composable test helper into my application, and if it works well, I'll release a library.

Permalink + Comments

strftime: table of locale-aware formatters in different locales

April 13, 2013 at 10:44 PM | Python | View Comments

I got curious about what the different locale-specific strftime and strptime formatting directives produced in different locales… So I built a little script which would genreate a table showing each of the different locale-aware formatters in all of the different locales which I've got installed!

The code used to generate this table can be found at: bitbucket.org/wolever/locale-table.

%a %A %b %B %x %X %p %c
af_ZA Tue Tuesday Aug August 08/16/1988 21:30:05 PM Tue Aug 16 21:30:05 1988
am_ET ማክሰ ማክሰኞ ኦገስ ኦገስት 16/08/1988 21:30:05 PM ማክሰ ኦገስ 16 21:30:05 1988
be_BY аў аўторак жні жніўня 16.08.88 21:30:05 pm аў 16 жні 21:30:05 1988
bg_BG Вт Вторник Авг Август 16.08.88 21:30:05 pm Вт 16 Авг 21:30:05 1988
ca_ES dim dimarts ago agost 16/08/1988 21:30:05 PM dim 16 ago 21:30:05 1988
cs_CZ út úterý srp srpna 1988/08/16 21:30:05 od út 16 srp 21:30:05 1988
da_DK Tir Tirsdag Aug August 16.08.1988 21:30:05 pm Tir 16 Aug 21:30:05 1988
de_AT Di Dienstag Aug August 16.08.1988 21:30:05 pm Di 16 Aug 21:30:05 1988
de_CH Di Dienstag Aug August 16.08.1988 21:30:05 pm Di 16 Aug 21:30:05 1988
de_DE Di Dienstag Aug August 16.08.1988 21:30:05 pm Di 16 Aug 21:30:05 1988
el_GR Τρι Τρίτη Αυγ Αυγούστου 16/08/1988 21:30:05 μμ Τρι 16 Αυγ 21:30:05 1988
en_AU Tue Tuesday Aug August 16/08/1988 21:30:05 pm Tue 16 Aug 21:30:05 1988
en_CA Tue Tuesday Aug August 16/08/1988 21:30:05 pm Tue 16 Aug 21:30:05 1988
en_GB Tue Tuesday Aug August 16/08/1988 21:30:05 pm Tue 16 Aug 21:30:05 1988
en_IE Tue Tuesday Aug August 16/08/1988 21:30:05 pm Tue 16 Aug 21:30:05 1988
en_NZ Tue Tuesday Aug August 16/08/1988 21:30:05 pm Tue 16 Aug 21:30:05 1988
en_US Tue Tuesday Aug August 08/16/1988 21:30:05 PM Tue Aug 16 21:30:05 1988
es_ES mar martes ago agosto 16/08/1988 21:30:05 PM mar 16 ago 21:30:05 1988
et_EE T teisipäev aug august 16.08.1988 21:30:05 T, 16. aug 1988. 21:30:05
eu_ES as. asteartea Abu abuztua 1988/08/16 21:30:05 p.m. 1988 - Abu - 16 as. 21:30:05
fi_FI Ti Tiistai Elo Elokuu 16.08.1988 21:30:05 pm Ti 16 Elo 21:30:05 1988
fr_BE Mar Mardi aoû août 16.08.1988 21:30:05 Mar 16 aoû 21:30:05 1988
fr_CA Mar Mardi aoû août 16.08.1988 21:30:05 Mar 16 aoû 21:30:05 1988
fr_CH Mar Mardi aoû août 16.08.1988 21:30:05 Mar 16 aoû 21:30:05 1988
fr_FR Mar Mardi aoû août 16.08.1988 21:30:05 Mar 16 aoû 21:30:05 1988
he_IL ג' שלישי אוג אוגוסט 16/08/88 21:30:05 PM 21:30:05 1988 אוג 16 ג'
hr_HR Ut Utorak Kol Kolovoz 16.08.1988 21:30:05 pm Ut 16 Kol 21:30:05 1988
hu_HU Ked Kedd Aug Augusztus 1988/08/16 21:30:05 du Ked Aug 16 21:30:05 1988
hy_AM Երք Երեքշաբթի Օգս Օգոստոս 16.08.1988 21:30:05 Երեքշաբթի, 16 Օգոստոս 1988 ի. 21:30:05
is_IS þri þriðjudagur ágú ágúst 16.08.1988 21:30:05 eh þri 16 ágú 21:30:05 1988
it_CH Mar Martedì Ago Agosto 16.08.1988 21:30:05 pm Mar 16 Ago 21:30:05 1988
it_IT Mar Martedì Ago Agosto 16.08.1988 21:30:05 pm Mar 16 Ago 21:30:05 1988
ja_JP 火曜日 8 8月 1988/08/16 21時30分05秒 PM 火 8/16 21:30:05 1988
kk_KZ сс сейсенбі там тамыз 16.08.1988 21:30:05 сейсенбі, 16 тамыз 1988 ж. 21:30:05
ko_KR 화요일 8 8월 1988/08/16 21시 30분 05초 PM 화 8/16 21:30:05 1988
lt_LT An Antradienis Rgp rugpjūčio 1988.08.16 21:30:05 An Rgp 16 21:30:05 1988
nl_BE di dinsdag aug augustus 16-08-1988 21:30:05 pm di 16 aug 21:30:05 1988
nl_NL di dinsdag aug augustus 16-08-1988 21:30:05 pm di 16 aug 21:30:05 1988
no_NO tir tirsdag aug august 16.08.1988 21:30:05 pm tir 16 aug 21:30:05 1988
pl_PL wto wtorek sie sierpnia 1988.08.16 21:30:05 wto 16 sie 21:30:05 1988
pt_BR Ter Terça Feira Ago Agosto 16/08/1988 21:30:05 Ter 16 Ago 21:30:05 1988
pt_PT Ter Terça Feira Ago Agosto 16.08.1988 21:30:05 Ter 16 Ago 21:30:05 1988
ro_RO Mar Marţi Aug August 16.08.1988 21:30:05 pm Mar 16 Aug 1988 21:30:05
ru_RU вт вторник авг августа 16.08.1988 21:30:05 вторник, 16 августа 1988 г. 21:30:05
sk_SK ut utorok aug august 16.08.1988 21:30:05 ut 16 aug 21:30:05 1988
sl_SI tor torek avg avgust 16.08.1988 21:30:05 pm tor 16 avg 21:30:05 1988
sr_YU уто уторак авг август 16.08.1988 21:30:05 уто 16 авг 21:30:05 1988
sv_SE Tis Tisdag Aug Augusti 16.08.1988 21:30:05 pm Tis 16 Aug 21:30:05 1988
tr_TR Sal Salı Ağu Ağustos 16/08/1988 21:30:05 PM Sal 16 Ağu 21:30:05 1988
uk_UA вт вівторок сер серпня 16.08.1988 21:30:05 вт 16 сер 21:30:05 1988
zh_CN 星期二 8 八月 1988/08/16 21时30分05秒 下午 二 8/16 21:30:05 1988
zh_HK 周二 8 8月 1988/08/16 21時30分05秒 下午 二 8/16 21:30:05 1988
zh_TW 周二 8 8月 1988/08/16 21時30分05秒 下午 二 8/16 21:30:05 1988
Permalink + Comments

South's frozen models are big liars

March 07, 2013 at 02:53 PM | Python, Django | View Comments

Consider South's orm object, http://south.readthedocs.org/en/latest/ormfreezing.html, which provides "frozen-in-time" versions of ORM models:

(Pdb++) from my_app.models import MyModel
(Pdb++) MyModel
<class 'my_app.models.MyModel'>
(Pdb++) orm['my_app.MyModel']
<class 'my_app.models.MyModel'>
(Pdb++) orm['my_app.MyModel'] == MyModel
False

Notice that the magic South frozen model appears to be in the same namespace as my real model: my_app.models!

This is at best stupid, and at worst potentially dangerous.

It makes perfect sense that South needs some mechanism for providing "frozen models" (ie, which match the database schema at the time of migration, of the current schema), but they are emphatically not the same as the real models (don't have any of the real methods, etc), so it should be made very clear that they are different! There are many ways this could be done, but one very simple step would be putting them in a different namespace, which makes it clear that these models are frozen, and references the migrations they belong to:

(Pdb++) orm['my_app.MyModel']
<class 'south.frozen_models.my_app.models_0003.MyModel'>

Additionally, this is a bad code smell which could potentially lead to bugs: because the "frozen model class" is lying about where it's from, any code which relies on the module path will do surprising things. In fact, it's such a big problem that pickle explicitly checks for this case:

(Pdb++) import pickle
(Pdb++) pickle.dumps(orm['my_app.MyModel'])
*** PicklingError: Can't pickle <class 'my_app.models.MyModel'>:
    it's not the same object as my_app.models.MyModel
Permalink + Comments

Why I deploy with rsync

January 18, 2013 at 11:24 PM | Uncategorized | View Comments

To deploy my application, I use a collection of bash scripts known lovingly as explode. At its core, explode uses rsync to copy my source tree exactly as it appears on my laptop up to the production server:

project="myproject"
src_dir="~/code/myproject"
target_dir="~/myproject-deploy"

ssh "$server" "
    /etc/init.d/$project stop;
    cd $target_dir;
    git commit -am 'pre-deploy commit';
"
rsync "$src_dir/" "$server:$target_dir/code/"
ssh "$server" "
    cd $target_dir;
    git commit -am 'post-deploy commit';
    /etc/init.d/$project start;
"

Now, at this point you're probably thinking:

  • Doesn't that mean that your deploy could contain files which aren't in source control?
  • What if you forget to pull before you deploy?

And those are definitely real problems.

But I've found that the popular alternative, deploying from the project's source control repository, has two significant drawbacks when deployments to environments other than production (staging, testing, demos, etc) are considered:

  • Pushing quick, experimental changes is annoying when a full commit to source control is required. You can often tell when a project deploys from source control because the commit log will contain clusters of commits with messages like "trying foo", "nope, that didn't work, trying bar", ... etc.
  • If the deploy script doesn't verify that there are no uncommitted changes to the local source code before deploying, the deployed code will be different from the local code... And if it does, it can make quick deploys frustrating.

Obviously these are not insurmountable problems, but I've found it easier to base my deployments on rsync and add sanity checks around production deployments ("working tree contains uncommitted changes... are you sure you want to deploy?") than it is to base my deployments on source control and add workarounds for deployments to non-production environments.

Now, to be clear: deplying with rsync isn't the be-all-and-end-all... It definitely has problems: rsyncing a source tree is significantly (significantly!) slower than a git push, and it's no where near as reproduceable as pulling from master. But it works well for me :)

Oh, and the git commit that's part of the deployment? While not necessarily related to rsync, I think it's worth mentioning too: the deployment's git repository contains the entire environment, including the Pyhton virtual environment! So if, for just about any reason, a deploy fails, git checkout HEAD^ will restore the deployment to a working state (the repository doesn't contain logs, the database, user data, though... As much fun as that would be, it would also be slightly impartial).

tl;dr: rsync makes it super easy to deploy to non-production environments, and deploying to production can be reasonably safe if a few sanity checks are performed.

Permalink + Comments

Anti-Pattern: Duck typing with try/except

November 29, 2012 at 06:46 PM | Python | View Comments

An anti-pattern I've often seen is using try/except incorrectly in an attempt at duck typing:

try:
    widget.frob()
except AttibuteError:
    # widget can't be frobbed
    do_something_else()

This is fundamentally broken because it will conceal bugs in the widget.frob() method (ie, if there's a bug in widget.frob() which causes an AttibuteError to be raised).

In this situation, the only safe way to use try/except would be:

try:
    widget_frob = widget.frob
except AttibuteError:
    # widget can't be frobbed
    do_something_else()
else:
    widget_frob()

This ensures that bugs in widget.frob() won't be accidentally hidden.

Of course, this code is no where near as pretty as the "incorrect" version, which is why I've come to believe that using try/except for duck typing could be considered an anti-pattern, and that hasattr is generally preferable (except in situations where performance is important, as try/except will almost certainly be faster):

if hasattr(widget, "frob"):
    widget.frob()
else:
    # widget can't be frobbed
    do_something_else()

And of course, all this goes without saying that a naked except: is always an absolutely terrible idea...

Permalink + Comments