strftime: table of locale-aware formatters in different locales

April 13, 2013 at 10:44 PM | Python | View Comments

I got curious about what the different locale-specific strftime and strptime formatting directives produced in different locales… So I built a little script which would genreate a table showing each of the different locale-aware formatters in all of the different locales which I've got installed!

The code used to generate this table can be found at: bitbucket.org/wolever/locale-table.

%a %A %b %B %x %X %p %c
af_ZA Tue Tuesday Aug August 08/16/1988 21:30:05 PM Tue Aug 16 21:30:05 1988
am_ET ማክሰ ማክሰኞ ኦገስ ኦገስት 16/08/1988 21:30:05 PM ማክሰ ኦገስ 16 21:30:05 1988
be_BY аў аўторак жні жніўня 16.08.88 21:30:05 pm аў 16 жні 21:30:05 1988
bg_BG Вт Вторник Авг Август 16.08.88 21:30:05 pm Вт 16 Авг 21:30:05 1988
ca_ES dim dimarts ago agost 16/08/1988 21:30:05 PM dim 16 ago 21:30:05 1988
cs_CZ út úterý srp srpna 1988/08/16 21:30:05 od út 16 srp 21:30:05 1988
da_DK Tir Tirsdag Aug August 16.08.1988 21:30:05 pm Tir 16 Aug 21:30:05 1988
de_AT Di Dienstag Aug August 16.08.1988 21:30:05 pm Di 16 Aug 21:30:05 1988
de_CH Di Dienstag Aug August 16.08.1988 21:30:05 pm Di 16 Aug 21:30:05 1988
de_DE Di Dienstag Aug August 16.08.1988 21:30:05 pm Di 16 Aug 21:30:05 1988
el_GR Τρι Τρίτη Αυγ Αυγούστου 16/08/1988 21:30:05 μμ Τρι 16 Αυγ 21:30:05 1988
en_AU Tue Tuesday Aug August 16/08/1988 21:30:05 pm Tue 16 Aug 21:30:05 1988
en_CA Tue Tuesday Aug August 16/08/1988 21:30:05 pm Tue 16 Aug 21:30:05 1988
en_GB Tue Tuesday Aug August 16/08/1988 21:30:05 pm Tue 16 Aug 21:30:05 1988
en_IE Tue Tuesday Aug August 16/08/1988 21:30:05 pm Tue 16 Aug 21:30:05 1988
en_NZ Tue Tuesday Aug August 16/08/1988 21:30:05 pm Tue 16 Aug 21:30:05 1988
en_US Tue Tuesday Aug August 08/16/1988 21:30:05 PM Tue Aug 16 21:30:05 1988
es_ES mar martes ago agosto 16/08/1988 21:30:05 PM mar 16 ago 21:30:05 1988
et_EE T teisipäev aug august 16.08.1988 21:30:05 T, 16. aug 1988. 21:30:05
eu_ES as. asteartea Abu abuztua 1988/08/16 21:30:05 p.m. 1988 - Abu - 16 as. 21:30:05
fi_FI Ti Tiistai Elo Elokuu 16.08.1988 21:30:05 pm Ti 16 Elo 21:30:05 1988
fr_BE Mar Mardi aoû août 16.08.1988 21:30:05 Mar 16 aoû 21:30:05 1988
fr_CA Mar Mardi aoû août 16.08.1988 21:30:05 Mar 16 aoû 21:30:05 1988
fr_CH Mar Mardi aoû août 16.08.1988 21:30:05 Mar 16 aoû 21:30:05 1988
fr_FR Mar Mardi aoû août 16.08.1988 21:30:05 Mar 16 aoû 21:30:05 1988
he_IL ג' שלישי אוג אוגוסט 16/08/88 21:30:05 PM 21:30:05 1988 אוג 16 ג'
hr_HR Ut Utorak Kol Kolovoz 16.08.1988 21:30:05 pm Ut 16 Kol 21:30:05 1988
hu_HU Ked Kedd Aug Augusztus 1988/08/16 21:30:05 du Ked Aug 16 21:30:05 1988
hy_AM Երք Երեքշաբթի Օգս Օգոստոս 16.08.1988 21:30:05 Երեքշաբթի, 16 Օգոստոս 1988 ի. 21:30:05
is_IS þri þriðjudagur ágú ágúst 16.08.1988 21:30:05 eh þri 16 ágú 21:30:05 1988
it_CH Mar Martedì Ago Agosto 16.08.1988 21:30:05 pm Mar 16 Ago 21:30:05 1988
it_IT Mar Martedì Ago Agosto 16.08.1988 21:30:05 pm Mar 16 Ago 21:30:05 1988
ja_JP 火曜日 8 8月 1988/08/16 21時30分05秒 PM 火 8/16 21:30:05 1988
kk_KZ сс сейсенбі там тамыз 16.08.1988 21:30:05 сейсенбі, 16 тамыз 1988 ж. 21:30:05
ko_KR 화요일 8 8월 1988/08/16 21시 30분 05초 PM 화 8/16 21:30:05 1988
lt_LT An Antradienis Rgp rugpjūčio 1988.08.16 21:30:05 An Rgp 16 21:30:05 1988
nl_BE di dinsdag aug augustus 16-08-1988 21:30:05 pm di 16 aug 21:30:05 1988
nl_NL di dinsdag aug augustus 16-08-1988 21:30:05 pm di 16 aug 21:30:05 1988
no_NO tir tirsdag aug august 16.08.1988 21:30:05 pm tir 16 aug 21:30:05 1988
pl_PL wto wtorek sie sierpnia 1988.08.16 21:30:05 wto 16 sie 21:30:05 1988
pt_BR Ter Terça Feira Ago Agosto 16/08/1988 21:30:05 Ter 16 Ago 21:30:05 1988
pt_PT Ter Terça Feira Ago Agosto 16.08.1988 21:30:05 Ter 16 Ago 21:30:05 1988
ro_RO Mar Marţi Aug August 16.08.1988 21:30:05 pm Mar 16 Aug 1988 21:30:05
ru_RU вт вторник авг августа 16.08.1988 21:30:05 вторник, 16 августа 1988 г. 21:30:05
sk_SK ut utorok aug august 16.08.1988 21:30:05 ut 16 aug 21:30:05 1988
sl_SI tor torek avg avgust 16.08.1988 21:30:05 pm tor 16 avg 21:30:05 1988
sr_YU уто уторак авг август 16.08.1988 21:30:05 уто 16 авг 21:30:05 1988
sv_SE Tis Tisdag Aug Augusti 16.08.1988 21:30:05 pm Tis 16 Aug 21:30:05 1988
tr_TR Sal Salı Ağu Ağustos 16/08/1988 21:30:05 PM Sal 16 Ağu 21:30:05 1988
uk_UA вт вівторок сер серпня 16.08.1988 21:30:05 вт 16 сер 21:30:05 1988
zh_CN 星期二 8 八月 1988/08/16 21时30分05秒 下午 二 8/16 21:30:05 1988
zh_HK 周二 8 8月 1988/08/16 21時30分05秒 下午 二 8/16 21:30:05 1988
zh_TW 周二 8 8月 1988/08/16 21時30分05秒 下午 二 8/16 21:30:05 1988
Permalink + Comments

South's frozen models are big liars

March 07, 2013 at 02:53 PM | Python, Django | View Comments

Consider South's orm object, http://south.readthedocs.org/en/latest/ormfreezing.html, which provides "frozen-in-time" versions of ORM models:

(Pdb++) from my_app.models import MyModel
(Pdb++) MyModel
<class 'my_app.models.MyModel'>
(Pdb++) orm['my_app.MyModel']
<class 'my_app.models.MyModel'>
(Pdb++) orm['my_app.MyModel'] == MyModel
False

Notice that the magic South frozen model appears to be in the same namespace as my real model: my_app.models!

This is at best stupid, and at worst potentially dangerous.

It makes perfect sense that South needs some mechanism for providing "frozen models" (ie, which match the database schema at the time of migration, of the current schema), but they are emphatically not the same as the real models (don't have any of the real methods, etc), so it should be made very clear that they are different! There are many ways this could be done, but one very simple step would be putting them in a different namespace, which makes it clear that these models are frozen, and references the migrations they belong to:

(Pdb++) orm['my_app.MyModel']
<class 'south.frozen_models.my_app.models_0003.MyModel'>

Additionally, this is a bad code smell which could potentially lead to bugs: because the "frozen model class" is lying about where it's from, any code which relies on the module path will do surprising things. In fact, it's such a big problem that pickle explicitly checks for this case:

(Pdb++) import pickle
(Pdb++) pickle.dumps(orm['my_app.MyModel'])
*** PicklingError: Can't pickle <class 'my_app.models.MyModel'>:
    it's not the same object as my_app.models.MyModel
Permalink + Comments

Why I deploy with rsync

January 18, 2013 at 11:24 PM | Uncategorized | View Comments

To deploy my application, I use a collection of bash scripts known lovingly as explode. At its core, explode uses rsync to copy my source tree exactly as it appears on my laptop up to the production server:

project="myproject"
src_dir="~/code/myproject"
target_dir="~/myproject-deploy"

ssh "$server" "
    /etc/init.d/$project stop;
    cd $target_dir;
    git commit -am 'pre-deploy commit';
"
rsync "$src_dir/" "$server:$target_dir/code/"
ssh "$server" "
    cd $target_dir;
    git commit -am 'post-deploy commit';
    /etc/init.d/$project start;
"

Now, at this point you're probably thinking:

  • Doesn't that mean that your deploy could contain files which aren't in source control?
  • What if you forget to pull before you deploy?

And those are definitely real problems.

But I've found that the popular alternative, deploying from the project's source control repository, has two significant drawbacks when deployments to environments other than production (staging, testing, demos, etc) are considered:

  • Pushing quick, experimental changes is annoying when a full commit to source control is required. You can often tell when a project deploys from source control because the commit log will contain clusters of commits with messages like "trying foo", "nope, that didn't work, trying bar", ... etc.
  • If the deploy script doesn't verify that there are no uncommitted changes to the local source code before deploying, the deployed code will be different from the local code... And if it does, it can make quick deploys frustrating.

Obviously these are not insurmountable problems, but I've found it easier to base my deployments on rsync and add sanity checks around production deployments ("working tree contains uncommitted changes... are you sure you want to deploy?") than it is to base my deployments on source control and add workarounds for deployments to non-production environments.

Now, to be clear: deplying with rsync isn't the be-all-and-end-all... It definitely has problems: rsyncing a source tree is significantly (significantly!) slower than a git push, and it's no where near as reproduceable as pulling from master. But it works well for me :)

Oh, and the git commit that's part of the deployment? While not necessarily related to rsync, I think it's worth mentioning too: the deployment's git repository contains the entire environment, including the Pyhton virtual environment! So if, for just about any reason, a deploy fails, git checkout HEAD^ will restore the deployment to a working state (the repository doesn't contain logs, the database, user data, though... As much fun as that would be, it would also be slightly impartial).

tl;dr: rsync makes it super easy to deploy to non-production environments, and deploying to production can be reasonably safe if a few sanity checks are performed.

Permalink + Comments

Anti-Pattern: Duck typing with try/except

November 29, 2012 at 06:46 PM | Python | View Comments

An anti-pattern I've often seen is using try/except incorrectly in an attempt at duck typing:

try:
    widget.frob()
except AttibuteError:
    # widget can't be frobbed
    do_something_else()

This is fundamentally broken because it will conceal bugs in the widget.frob() method (ie, if there's a bug in widget.frob() which causes an AttibuteError to be raised).

In this situation, the only safe way to use try/except would be:

try:
    widget_frob = widget.frob
except AttibuteError:
    # widget can't be frobbed
    do_something_else()
else:
    widget_frob()

This ensures that bugs in widget.frob() won't be accidentally hidden.

Of course, this code is no where near as pretty as the "incorrect" version, which is why I've come to believe that using try/except for duck typing could be considered an anti-pattern, and that hasattr is generally preferable (except in situations where performance is important, as try/except will almost certainly be faster):

if hasattr(widget, "frob"):
    widget.frob()
else:
    # widget can't be frobbed
    do_something_else()

And of course, all this goes without saying that a naked except: is always an absolutely terrible idea...

Permalink + Comments

Adventures in X.509: The Utterly Ignored nameConstraints

April 08, 2012 at 09:16 PM | OpenSSL, MintChip | View Comments

In yesterday's post looking at MintChip's hosted API's crypto, I noticed that MintChip's certificate authority does not have any name constraints. This was surprising, because this seemed like a simple and obvious issue with an otherwise well designed system... But after some experimentation, it seems that OpenSSL simply ignores nameConstraints, so they would be mostly worthless even if they were used.

Adding nameConstraints to MintChip's CA

This can be seen by adding some nameConstraints to MintChip's certificate authority and re-signing it using a new, trusted, certificate authority.

Note that all files referenced here can be found at /uploads/adventures_in_x509_ignored_nameconstraints/.

A new certificate authority is created and the add_name_constraints.py to add nameConstraints to MintChip's CA's certiificate, creating mintchip_with_constraints.crt, which will be signed by the new CA:

$ openssl req -new -x509 -keyout trusted_ca.pem -out trusted_ca.pem -nodes
...
Organizational Unit Name (eg, section) []: New Trusted Authority
...
$ ./add_name_constraints.py trusted_ca.pem mintchip_ca.crt > mintchip_with_constraints.crt
$ openssl x509 -noout -text -in mintchip_with_constraints.crt
Data:
    ...
    Issuer: OU=New Trusted Authority
    Subject: CN=Remote MintChip Certificate Authority, OU=Remote MintChip, O=Royal Canadian Mint, C=CA
    ...
    X509v3 extensions:
        ...
        X509v3 Name Constraints: critical
            Permitted:
              DNS:does_not_exist.com

Testing the Constraints

The constraints can now be tested using OpenSSL's s_client:

$ cat trusted_ca.pem mintchip_with_constraints.crt > ca_list.crt
$ openssl s_client -CAfile ca_list.crt -connect remote.mintchipchallenge.com:443
CONNECTED(00000003)
depth=2 /C=AU/ST=Some-State/O=Internet Widgits Pty Ltd/OU=New Trusted Authority
verify return:1
depth=1 /CN=Remote MintChip Certificate Authority/OU=Remote MintChip/O=Royal Canadian Mint/C=CA
verify return:1
depth=0 /CN=remote.mintchipchallenge.com/OU=Remote MintChip Server/O=Royal Canadian Mint/C=CA
verify return:1
...
SSL-Session:
    ...
    Verify return code: 0 (ok)
---

Note that:

  1. Our "New Trusted Authority" is used to verify MintChip's Certificate Authority (first six lines).
  2. The SSL session is successfully verified (the last line).

This suggests that OpenSSL is not checking name constraints.

Double Checking

To double check that our certificat authority is actually being consulted, the same test can be run, except using only the mintchip_with_constraints.crt certificate:

$ openssl s_client -CAfile mintchip_with_constraints.crt \
    -connect remote.mintchipchallenge.com:443
CONNECTED(00000003)
depth=1 /CN=Remote MintChip Certificate Authority/OU=Remote MintChip/O=Royal Canadian Mint/C=CA
verify error:num=2:unable to get issuer certificate
issuer= /C=AU/ST=Some-State/O=Internet Widgits Pty Ltd/OU=New Trusted Authority
verify return:0
...
SSL-Session:
    ...
    Verify return code: 2 (unable to get issuer certificate)
---

The verify now fails, so it's fairly certain that the certificate chain is working as expected.

Adding dirName nameConstraints

Applications conforming to this profile MUST be able to process name constraints that are imposed on the directoryName name form and SHOULD be able to process name constraints that are imposed on the rfc822Name, uniformResourceIdentifier, dNSName, and iPAddress name forms.

On closer inspection of RFC 5280, it turns out that applications are only required to check name constraints which are imposed on directory names.

This poses a slight challenge, as (for various reasons) pyOpenSSL cannot create a nameConstraints extension which imposes a dirName constraint[0].

The OpenSSL command line tools can be used to create a new certificate which does contain nameConstraints on a dirName (see the definition in name_constraints_on_dirNames.cfg), then pyOpenSSL can load that certificate and copy the constraints into the target:

$ openssl req -new -x509 -nodes -config name_constraints_on_dirNames.cfg \
>   -out dirName_constraints.crt
...
$ ./add_name_constraints.py trusted_ca.pem mintchip_ca.crt \
>   dirName_constraint.crt > mintchip_with_dirName_constraints.crt
...
$ openssl x509 -noout -text -in mintchip_with_dirName_constraints.crt
Certificate:
    ...
    X509v3 extensions:
        ...
        X509v3 Name Constraints: critical
            Permitted:
              DirName: CN = bad_common_name

Nifty!

[0]: OpenSSL expects the value of the dirName=... to be a name which is looked up in the configuration database (ex, nameConstraints = permitted;dirName=some_section_), but pyOpenSSL does not provide a configuration database.

Testing the dirName Constraint

Testing this new certificate with s_client:

$ cat trusted_ca.pem mintchip_with_dirName_constraints.crt \
>   > ca_list_with_dirName_constraints.crt
$ openssl s_client -CAfile ca_list_with_dirName_constraints.crt \
>   -connect remote.mintchipchallenge.com:443
CONNECTED(00000003)
depth=2 /C=AU/ST=Some-State/O=Internet Widgits Pty Ltd/OU=New Trusted Authority
verify return:1
depth=1 /CN=Remote MintChip Certificate Authority/OU=Remote MintChip/O=Royal Canadian Mint/C=CA
verify return:1
depth=0 /CN=remote.mintchipchallenge.com/OU=Remote MintChip Server/O=Royal Canadian Mint/C=CA
verify return:1
...
SSL-Session:
    ...
    Verify return code: 0 (ok)
---

Still no love. It sure looks like OpenSSL simply ignores the nameConstraints field on certificate authorities.

Post Script

On Twitter, Dan Kaminsky was kind enough to "confirm" my suspicions:

And suggests:

tl;dr

OpenSSL does not check nameConstraints. Yay.

Permalink + Comments