Code Kills

A little while ago, I started noticing some odd behaviour in DrProject. Every once and a while, a stack trace complaining about primary key violations in the session table would come up when trying to view the list of tickets with Firefox. I ticketed it, but didn't think much of it until it started happening more frequently (and in production).

First, a bit of background. When ever a user views a list of tickets in DrProject, it writes that list to a session variable. This is so that the "next ticket" and "previous ticket" links (in the single-ticket view) do the RightThing™. The code that handles that is something like this:

s = SessionVariable.query(session_id=id, name="ticket_list")
if s:
   s.value = new_list
else:
   s = SessionVariable(session_id=id, name="ticket_list", value=new_list)
s.flush_to_database() 

Looks fine, right? Well, not if two threads with the same session id try to update the ticket list within 1/100th of a second of each other... But that's not really a problem, is it? Unfortunately that's exactly what Firefox is doing.

Firefox decides that she doesn't like the first connection (0), so she kills it off and starts again. The full dump is also available.

When the "View Tickets" link is clicked, this is what Firefox does:

1. Opens one connection
2. Sends headers
3. Waits 5 millionths of a second
4. Closes the connection (see 0)
5. Opens a new connection and continues as normal (see 1)

Wonderful.

As the two requests come in, neither thread can see a ticket_list session variable, so both try to generate one... But when the second call to flush_to_database() comes in, the database raises an IntegrityError because the data is already there! This thread sends a stack trace back up to the user. The other thread happily returns, oblivious to the problem.

Now, this is where things get a little bit more interesting. Because all this has happened so quickly, there is no way to know which thread corresponds to which request (the one that Firefox has closed or the one that will make it back to the user). If we're lucky, Firefox will get the stack trace and the user will see a nice list of tickets... But that's hardly a safe design principle.

So how do we fix this?

1. Give each request a unique ID.
   If two requests with the same ID come in, you just ignore the all but the first. But this doesn't work because you may be ignoring the request going to the user.

2. Figure out why Firefox is making two requests.
   This turns out to be Really Hard. Greg sent an email off to David Humphrey, who suggested that we make sure there isn't some Java Script trickery going on. After a bit of digging around, nothing came up. With JavaScript turned off, everything was fine. When I turned JavaScript on but commented out all the >script< tags, things were still exploding. Crap.

3. Wrap the code in a try/catch block, and just ignore any IntegrityErrors that come up.

Well, up until about five minutes ago, the plan was to go with option 3... But as I was writing this, I realized there was a very remote possibility that some onClick trickery could be going on. And guess what?

Yet again have I forgotten one of the most fundamental rules of programming: printf isn't broken.

Update

I have not had a chance to get in front of the code yet, but when I do the solution will be to remove the offending JavaScript.

Alex originally added it so that the entire row (<tr>) could be clicked like a hyperlink... But it had this unfortunate side effect (and, Dmitri added, you can't open these pseudo-links in new tabs).

After a bit of Googling, I can't seem to find any HTML-only method of turning entire table rows into hyperlinks... So, unless anyone has an insight into this, we will just need to live with old-fashioned text-only hyperlinks.

Update 2

I was talking with Andrew Louis today, and he pointed me to this article. It shows how to do exactly the same thing the JavaScript did before, just in nice "clean" CSS (at least, it's about as "clean" as CSS can be). It felt good to rip out a huge block of JavaScript and nested tables

I got the game BioShock when it came out and had no issues when I started out and happily started playing through the game. Then as usual, I started screwing with things that I shouldn't have and updated my BIOS just so I could have the latest version. The update worked fine and everything was alright until I tried to continue with BioShock upon which I discovered there was no more sound in game. I have a MSI P965 Platinum motherboard and currently use its onboard sound controller. After rolling back the BIOS a few times I determined the breaking version was the 1.03 which has a change list indicating changes related to the north and south bridge were made and these changes seem to have screwed up my sound. Reverting to 1.02 fixed all problems and I was able to finish the game. After the breaking update I noticed that Vista (Home Premium x64) recognized sound as a new device. Maybe new drivers got screwed up, but I'm not sure. I didn't explore it in much detail. The sound worked in Windows and for the loading of the game, but once DirectSound (or whatever it is now called in DirectX10) started, the sound was gone. I didn't compare with other games though so I can't comment on how they handled it.

Hopefully the search engines will catch this and help anyone with similar problems.

I recently reformatted my desktop and so had to reinstall and in the process updated to newer versions of things. Since I do play with Windows development I reinstalled the latest platform SDK (Feb 2007) now known as the Microsoft® Windows® Software Development Kit Update for Windows Vista™. In an attempt to put off doing some homework I looked around through what was included in this latest SDK and was happy to find full x86/x64 native compilers and cross-compilers. I'm just a student who doesn't make any money with the code I do so I just have been using the Express Versions of Visual Studio 2005 which are free but are limited. They don't include interesting features to play with like OpenMP and 64-bit compilers. Now with a new 64-bit computer I was happy to have 64-bit windows compilers to play with. Anyways, I discovered that the compilers have /openmp flags enabled and seemed to work, although the program wouldn't run thanks to some missing dll's.

The next step to getting it working was to find the Visual Studio 2005 redist pack that contains the dlls for running Visual C++ programs such as the standard library DLL and of course the OpenMP runtime DLL. Now the compilers with this SDK are 14.00.50727.762 which are aligned with Visual Studio 2005 SP1 so we need the SP1 redist pack. The redist pack is available here (x86) or here (x64). This included the necessary vcomp.dll (The OpenMP runtime dll for Visual Studio 2005 SP1) but my test program still couldn't find it.

A bit more searching revealed that I was supposed to #include which would set up the manifest for side-by-side assemblies (Here is a brief description of the side-by-side assemblies thing). Running it again... Success! The test program ran. A quick check of the assembly code generated revealed it did in fact generate the OpenMP code.

Now to see if I could integrate at least partially with the Visual C++ Express IDE. It seems the latest Visual C++ 2005 Express edition has the /openmp option availible in the project options so I just needed to make sure that it was enabled. The one thing to make sure of when setting up is the Windows SDK is that the VC/Include directory is included in addition to Include (same goes for Lib). Compiling and running and it all seems to work still.

After playing around for a bit I discovered one big caveat is that there is no debug version of the OpenMP runtime available with this setup so we'll need some trickery so that omp.h doesn't try to set us up to link to it when the _DEBUG flag is set. It seems that this isn't enough to get program to run in debug mode. When OpenMP is enabled this way, even if we use this preprocessor stuff so that the non-debug version will be linked, if we also link to a debug version of the C/C++ runtime things will break. This means you will need to disable openmp when using debug versions of the CRT.

#ifndef _DEBUG
    #include <omp.h>
#endif /* _DEBUG */
 

What is OpenMP?

OpenMP is a specification of langauge extensions that allow a programmer to specify parts of the program that can be parallelized and the rules that need to be followed for correctness. Work is dynamically assigned to worker threads that automatically created by the OpenMP runtime. This allows data parallelism to be dealt with, without the need to manage threads and resources. I don't know much about it as I'm just starting to play with it so I'll let <insert search engine of choice> tell you more about it.
(Recent versions of GCC should also support OpenMP directly.)