The Bride of Progress Report on WordPress-pg 1.5

More PHP and SQL…

$id_result = $wpdb->get_row("SHOW TABLE STATUS LIKE '$wpdb->posts'");
$post_ID = $id_result->Auto_increment;

…then after 20 more lines…

$postquery ="INSERT INTO $wpdb->posts
(ID, …) VALUES ('$post_ID', …)";

This is a race condition. While being formally defined by FOLDOC as “anomalous behavior due to unexpected critical dependence on the relative timing of events”, in this case it can be more easily thought of as a race between two individuals trying to do the same thing. A pair of people racing to make it to a revolving door when the door will only fit one of them. A group of cars racing to make the ferry when only one more car will fit on the boat. That sort of idea…

This code replaced old code that was doing something similarly vulnerable to a race condition. Except that the old code also protected against incidents where the auto_increment counter is out of touch with reality somehow. (More weirdness with auto_increment is documented at Ian Barwick’s MySQL Gotchas page.) In both the old and new code, we’re determining what the new post id is going to be, then going off to do other things, then we’re inserting the post with that new post id. That is unless someone else who got that new post id the same way we did, did the other things, and then did the insert before we got our chance, in which case there is an error and the post creation fails (with an error that the average user is not going to understand since there is nothing looking for this sort of failure). There can be only one (successful post with a given id).

Frankly, I don’t actually know if there is a way to handle this safely in MySQL without either recognizing the error that comes back and trying again with a different ID that (hopefully) will work or just locking the table through the whole process. Both of which are far from elegant. Luckily, the frequent posting that would trigger this bug isn’t likely with your average blog and the commenting scheme uses different code which doesn’t need to obtain the comment id ahead of time.

In PostgreSQL, there are utility functions for dealing with the sequences used for auto-incrementing that let you get the next value safe from possible collisions or races. The downside to this approach is that if your transaction fails for some other reason the value you received will never be used by anyone. However since it is an internal id number that is never actually visible to the end users in most cases, it doesn’t really matter if there are gaps in the ids. There will be gaps in the ids if you delete posts anyway.

All we have to do for the PostgreSQL port is run this query instead:

SELECT next_val('${wpdb->posts}_id_seq')

With this we wind up both safe from the race condition for obtaining the next post id and with a very elegant looking solution to the potential problem. I’m sure those guys in Sweden will get around to implementing a feature akin to this soon. Maybe it’s there and I just haven’t heard of it yet.

Some editorializing follows, please cover your eyes if discussion about censorship by corporations bothers you…

Since this writing could be taken by some as “disparaging” the MySQL product, I’m apparently not authorized to use any MySQL AB’s trademarks in this blog entry, including the trade name “MySQL”. Don’t believe me? Read their rather draconian trademark policy for yourself. Good thing this country still has fair use laws, so I don’t need authorization from the manufacturer to write critical evaluations of things.

Progress Report on WordPress-pg 1.5

Here’s some SQL…

SELECT meta_key, meta_id FROM wp_postmeta GROUP BY meta_id ORDER BY meta_id DESC LIMIT 10;

This is just one of the many issues I’ve been working through in porting this code. Initial table creation is now successful and the sample post is populated and at least partially displayed. Editing is not working, new posts aren’t working, new pages aren’t working, new comments aren’t… well, you get the idea. At least I’m making forward progress though…

Let’s get back to that snippet at the top. Apparently in the latest revision of WordPress the authors decided that adding lots of “GROUP BY some-primary-key” statements would make things better. Now in ordinary database land, if you are “grouping by” some field, you have to give the database instructions on how to aggregate the other fields that you want displayed. In the above example, there may be 3 or 4 rows of meta_keys to choose from. As you can see above the database is given no instructions on which one of those 3 or 4 rows to use. PostgreSQL treats this as a malformed query and returns an error. MySQL in fairly typical fashion for it, just picks an arbitrary row to use. I think it’s using the first row in the table that it happens to come across during your query, but I could be wrong. The point is that it is not well-defined, and in ordinary database land, not well-defined behavior is considered harmful.

So, to make progress I basically have to strip all these GROUP BY statements back out again. And honestly, I’m unable to see what these statements accomplish. The code is “grouping by” a primary key sequence on a given table in all the cases I’ve happened across so far. By definition there should only be one row with a given primary key. But perhaps that’s only true in ordinary database land…

WordPress-Pg 1.2.2 in CVS

The WordPress-Pg project source code in CVS is up to date with WordPress 1.2.2 as of last Saturday. This is going to be of little consolation to people that want the latest WordPress release (1.5) which arrived during this week.

Given the larger scope of changes for the 1.5 release it may take a little longer to get things together. I’m also going to have to continue to fight the temptation to make improvements in the places where WordPress’s performance is lackluster. It isn’t clear to me from the release notes if they actually managed to do something about the volume of queries needed to render a page or not. A lot of it may be a result of being crippled by MySQL’s limitations as well (for instance, it can only use one index per table in each query), so they may not be able to do anything to help it.

In any case, if you’re interested in WordPress-pg 1.2.2 you can check it out from the Sourceforge anonymous CVS by following the directions at Sourceforge and adding -r WPPG_1_2_1 right after “co” in the checkout command.

New Pictures in the Gallery

We have some new pictures online in the gallery this evening. A month of pictures have built up and I’ve finally gotten them all online. The pictures include a New Year’s Day trip to the Mercer Slough in Bellevue, flood waters in the Snoqualmie Valley (including some nice pictures of Snoqualmie Falls with flood waters) and some interesting pictures that resulted from me fooling around with a new 50mm lens I got for Christmas.

As with everything else in the gallery, these were shot by either Eileen or I and we haven’t done any post-processing to clean the images up at all. There’ll be problems with levels and white balance and other such technical minutæ, but we hope you won’t hold it against us and enjoy the pictures as much as we do.