21 March 2005

The Return of Progress Report on WordPress-pg 1.5


I spent some quality time in a local coffee shop this evening pounding out more work on the port. Without further rambling here is the good and bad side of where we’re at now:


  • Installs from scratch works in my test environment

    • The logo image on the install page depends on a remote image, probably should get fixed to be local

    • The table creation statements in step 2 interleave wrong in my browser

    • There are some errors thrown by the index manipulation code in step 2

    • Haven’t tested an upgrade scenario of any sort (this probably won’t work)

    • Haven’t tested an import scenario of any sort (this certainly won’t work)



  • Creating new posts works

  • Creating new comments works

    • Comment status code is probably broken (badness related to comment_approved growing a new “SPAM” status which I’ve wound up implementing using NULL since this was made a boolean in the 1.2.x ports)



  • Creating new categories works

  • Creating new links works

  • The front page as well as individual post pages display correctly

  • The admin pages display mostly correct

    • There is some weirdness with the categories list in Manage Categories where root items are displayed twice except for the id 0 “Uncategorized” entries.



  • I’m seeing some database error messages on creating a post but can’t determine where they are coming from due to wp-db being obtuse and various bits of WordPress code turning error reporting off for no good reason

  • Haven’t even begun to look at xmlrpc.php yet. Making that go seems like it is going to be a bit of an adventure


  • Got the posts and categories displaying correctly on the end user parts

  • Fixed places where LIMIT M,N needed to become LIMIT N OFFSET M for PostgreSQL

  • Fixed more badness where “SHOW TABLE STATUS” was used to get the auto_increment counter for new categories, which resulted in new category creation working properly.

  • Default post category is now the special “Uncategorized” category. Some weird stuff was happening where the post_category would be 0 but wp_post2cat would have 1 in it for some reason


  • Cleanup database error messages to be able to know when things go wrong without tailing PostgreSQL’s logs

  • Make install.php use local images only

  • Fix install.php step 2 to use reindex rather than dropping and recreating indexes (which may be necessary for MySQL but not us)

  • Figure out and fix what is throwing the SQL syntax errors in post.php on “save as draft”

  • Fix the Category List order strangeness in wp-admin/categories.php

  • Change the default links in the install to not include every person who ever wrote a line of WordPress code and instead point to the project pages and useful resources like the WordPress Codex.

  • Change the default hello world comment and text to have some “Lorem Ipsum” copy in them to fill out space better.

So I’m going to be checking this all into CVS in a few minutes. I also took the time to add my thoughts about database abstraction approaches on the Using Alternative Databases page over on the WordPress Codex wiki. The WordPress-specific abstraction layer is going to work out best in the long haul. Highly generic abstraction layers for database access generally do poorly from optimization, performance and ease of coding points of view. And as the time spent on the 1.5 port demonstrates, the existing database code is a total fiasco for trying to add new databases to.


17 March 2005

Brave New VoIP World


We’ve made the plunge into the VoIP (Voice over IP: transmitting telephone calls over the Internet) world. After a few months of paying $20 a month on long distance I figured there must be better way. Our local phone company’s best plan without a monthly fee is 18¢ per minute. AT&T has a no monthly fee plan for 10¢ per minute, except that you pay a much higher rate for in-state long distance. MCI doesn’t even offer a no monthly fee plan any more.

Enter Internet long distance. A variety of VoIP companies offer long distance to the anywhere in the US, Canada, and most of western Europe for 2¢ per minute or less. No monthly fee. The only catch is that you have to get your call onto the internet. There are services that bundle a whole telephone line over the internet with long distance, a new phone number, and an interface box. These services also generally cost at least $20 per month. Worse yet, being a DSL customer out in the sticks, we can’t get DSL without a plain old telephone line. No use paying for two phone lines.

But you aren’t stuck with a package deal, you can always get your own interface box and buy your own long distance and still use the existing phone line you already have. In our case we got a Sipura SPA-3000 from AsteriskMall and setup Asterisk on a spare computer. With some amount of configuration work, we’ve got outgoing local calls still traversing the POTS line, outgoing long distance going over the Internet to our VoIP long distance provider (saving us 8-16¢ per minute), and incoming calls coming in over the POTS line. There are also some special numbers we can use to make free VoIP calls to other folks on the Internet like IAXtel’s 1-700 numbers and Free World Dialup. Free World Dialup in particular has free calling to anyone with the most popular bundle providers (Packet8, Vonage, etc.).

In the long run, our investment in the interface box will pay for itself in long distance savings in half a year. And since we still have our normal phone line which the interface box will fall back to automatically when there is a power outage we still get the E911 service, even when the power and/or Internet connection are not working.

This isn’t to say that this hasn’t been a bit of an adventure. The SPA-3000 has over one hundred configurable parameters and getting it configured correctly involved a lot of trial and error. (By the way, thanks to our parents who have been very kind about our phone calls as we try and get things working. “Can you hear me now?”) Voice quality has been fairly good though. The delay hasn’t been quite as bad as I had expected. It’s slightly better delay-wise than satellite calls across the Atlantic, but this depends on how Internet traffic is between our home and


  • Calls going out the POTS line are too quiet for both parties

    • Solved by increasing the PSTN Line “SPA to PSTN Gain” and “PSTN to SPA Gain” settings to 5 (dB presumably)



  • Calls going out the VoIP long distance have an echo for the people we call

    • Possibly solved by setting SIP “RTP Packet Size” to 0.010 (seconds)



  • Calls either way on the POTS line can’t make DTMF tones (touchtones) in call

    • No resolution yet



  • Calls either way on the POTS line rarely have an echo back to our phone when both sides are talking

    • No resolution yet


There is still a long ways we can go with this too. Now that we have Caller ID incoming calls can get special treatment depending on who they are and we can get some more powerful voice mail service as well.

Your call is important to us. Press 1 to learn about why paying for long distance service from the old communication monopolies is going to be obsolete in a dozen years…


04 March 2005

The Bride of Progress Report on WordPress-pg 1.5


More PHP and SQL…

$id_result = $wpdb->get_row("SHOW TABLE STATUS LIKE '$wpdb->posts'");
$post_ID = $id_result->Auto_increment;

…then after 20 more lines…

$postquery ="INSERT INTO $wpdb->posts
(ID, …) VALUES ('$post_ID', …)";

This is a race condition. While being formally defined by FOLDOC as “anomalous behavior due to unexpected critical dependence on the relative timing of events”, in this case it can be more easily thought of as a race between two individuals trying to do the same thing. A pair of people racing to make it to a revolving door when the door will only fit one of them. A group of cars racing to make the ferry when only one more car will fit on the boat. That sort of idea…

This code replaced old code that was doing something similarly vulnerable to a race condition. Except that the old code also protected against incidents where the auto_increment counter is out of touch with reality somehow. (More weirdness with auto_increment is documented at Ian Barwick’s MySQL Gotchas page.) In both the old and new code, we’re determining what the new post id is going to be, then going off to do other things, then we’re inserting the post with that new post id. That is unless someone else who got that new post id the same way we did, did the other things, and then did the insert before we got our chance, in which case there is an error and the post creation fails (with an error that the average user is not going to understand since there is nothing looking for this sort of failure). There can be only one (successful post with a given id).

Frankly, I don’t actually know if there is a way to handle this safely in MySQL without either recognizing the error that comes back and trying again with a different ID that (hopefully) will work or just locking the table through the whole process. Both of which are far from elegant. Luckily, the frequent posting that would trigger this bug isn’t likely with your average blog and the commenting scheme uses different code which doesn’t need to obtain the comment id ahead of time.

In PostgreSQL, there are utility functions for dealing with the sequences used for auto-incrementing that let you get the next value safe from possible collisions or races. The downside to this approach is that if your transaction fails for some other reason the value you received will never be used by anyone. However since it is an internal id number that is never actually visible to the end users in most cases, it doesn’t really matter if there are gaps in the ids. There will be gaps in the ids if you delete posts anyway.

All we have to do for the PostgreSQL port is run this query instead:

SELECT next_val('${wpdb->posts}_id_seq')

With this we wind up both safe from the race condition for obtaining the next post id and with a very elegant looking solution to the potential problem. I’m sure those guys in Sweden will get around to implementing a feature akin to this soon. Maybe it’s there and I just haven’t heard of it yet.

Some editorializing follows, please cover your eyes if discussion about censorship by corporations bothers you…

Since this writing could be taken by some as “disparaging” the MySQL product, I’m apparently not authorized to use any MySQL AB’s trademarks in this blog entry, including the trade name “MySQL”. Don’t believe me? Read their rather draconian trademark policy for yourself. Good thing this country still has fair use laws, so I don’t need authorization from the manufacturer to write critical evaluations of things.


23 February 2005

Progress Report on WordPress-pg 1.5


Here’s some SQL…

SELECT meta_key, meta_id FROM wp_postmeta GROUP BY meta_id ORDER BY meta_id DESC LIMIT 10;

This is just one of the many issues I’ve been working through in porting this code. Initial table creation is now successful and the sample post is populated and at least partially displayed. Editing is not working, new posts aren’t working, new pages aren’t working, new comments aren’t… well, you get the idea. At least I’m making forward progress though…

Let’s get back to that snippet at the top. Apparently in the latest revision of WordPress the authors decided that adding lots of “GROUP BY some-primary-key” statements would make things better. Now in ordinary database land, if you are “grouping by” some field, you have to give the database instructions on how to aggregate the other fields that you want displayed. In the above example, there may be 3 or 4 rows of meta_keys to choose from. As you can see above the database is given no instructions on which one of those 3 or 4 rows to use. PostgreSQL treats this as a malformed query and returns an error. MySQL in fairly typical fashion for it, just picks an arbitrary row to use. I think it’s using the first row in the table that it happens to come across during your query, but I could be wrong. The point is that it is not well-defined, and in ordinary database land, not well-defined behavior is considered harmful.

So, to make progress I basically have to strip all these GROUP BY statements back out again. And honestly, I’m unable to see what these statements accomplish. The code is “grouping by” a primary key sequence on a given table in all the cases I’ve happened across so far. By definition there should only be one row with a given primary key. But perhaps that’s only true in ordinary database land…