Data Validation Rant

I just signed up to renew my ACM membership for the first time since I was a student back at WWU. I have to admit to feeling kind of bad about having finally caved in to the direct mail they've been bombarding me with, but it did offer a reasonable discount. So I'm busy filling out their online form that asks for all sorts of information they can use to direct additional marketing at me and I click the "continue" button, but alas I cannot continue.

So why can I not continue? Everything I entered is correct, I double-checked. It turns out I put a dash in my zip+4 code, and for them that's an error. The US Postal Service disagrees, and in fact insists that xxxxx-xxxx is the "standard format" for a zip code. One would hope that the ACM of all organizations could figure out how to validate data without looking stupid, but apparently that's too much.

This is a real frustration in lots of other online settings as well. There are quite a few online merchants that refuse to process your payment unless you leave the spaces out of your credit card number. They display stern warnings like "Important: Please do not put spaces or dashes between credit card numbers." If they can't even figure out how to remove extraneous spaces or dashes from the information I give them, should I be trusting them to get the rest of my order correct and not have my personal information stolen? Dealing with that input is maybe 4 lines of code if you're trying to make it hard. It's usually not the merchant's fault, they're just buying some third party service. Any service that merchants are having to pay for that can't manage to get this right should be publicly shamed for their lack of competence.

Email addresses are also apparently a tricky beast for people who are building online tools. There are a lot of characters that are valid in an email address that various online services tend to choke on. There are numerous examples online disguised as "how-tos" that are actually "how-not-to-dos". The example I'm picking on in this case doesn't accept addresses with a + in them. Addresses like learn+to+read@rfc2822.int are in fact perfectly legitimate email addresses, as are addresses with hyphens, underscores, carets, tildes, equals and a lot of other somewhat obscure characters. Mail servers seem to deal successfully with these things, one would hope most
web application writers could cope with it too.

Come on all you folks that are building tools for the web, it's not hard to get these things right by just being little bit more pragmatic about it.

Moved to Blogger

As you may have noticed, I’ve moved the blog part of the site over here to Blogger’s new custom domain feature. Some old links to the blog pages may not work any more, but I've tried to get the majority of them to redirect correctly. My old custom hacked up Typo install was getting frayed around edges and its easier to let a free service handle this stuff for me as long as I can extract my data later.

It turns out that it's not too hard to take the data from Typo articles by getting them by id number and then finagle it into something that the Blogger API will take and do the migration programmatically. I did have to clean up the XHTML in places where tags hadn’t quite lined up, but it wasn't all that much work.

BOM Shelter: MoAB 5, 8, 15 Permissions Fix

Mac OS X, and a number of programs by third parties, have some risky permissions by default. I’ve taken the work I did a couple weeks ago and updated it to cover over more of these problems (MoAB days 5, 8, and 15). It also has a new more clever name: bom-shelter.py (sig)

To get this script, simply save the bom-shelter.py link to your disk.

To verify the script you get is one I wrote, you can download the signature and gpg --verify bom-shelter.py if you happen to have GnuPG installed and have a reason to trust my public key (0x4185664C).

To use this script, you must, from an admin account, run sudo python bom-shelter.py in Terminal, iTerm, or some other reasonable equivalent.

This script does the following for each of the MoAB advisories listed above:
#5: The permissions on BOM files are made more secure and /Library/Receipts (and important descendants) get a sticky bit to prevent shenanigans.

#8: /Library/Frameworks gets a sticky bit to prevent potential adversaries from being able to replace components that Application Enhancer runs as root inside Application Enhancer.framework.

#15: The three setuid root programs that can be overwritten by members of the “admin” group in /Applications/Utilities mentioned in the advisory are changed to not be “admin”-writable. This is also done to /Applications/System Preferences.app/Contents/Resources/installAssistant which has a similar vulnerability.

For all of these things, the script also edits the BOM files in /Library/Receipts to ensure that if you “repair permissions” on your disk these vulnerabilities will not reappear. The BOM file format is not very well documented so these edits may or may not work for you, but they should not corrupt the file. The editing function is careful to only change values if they are what is expected, otherwise it’ll print a warning and not make a change. Backup versions of your BOM files are saved as part of this process.

If you happen to have Application Enhancer installed, in order to secure your machine with Landon Fuller’s awesome MoAB Fixes or any other reason, please take the time to secure /Library/Application Enhancers outside your home directory and ~/Library/Application Enhancers inside your home directory. Malicious code can write things there without your permission and if Application Enhancer uses those patches without asking you, it might make you sad.

Counting Trigrams for Fun

There is a thread over at the forums for the xkcd comic strip with a puzzle game to find trigrams. To help out my brain which has been full of number-related work lately, I whipped up a Python script “trigramtastic.py” to help find more challenging trigrams for the game. Feeding it things like /usr/share/dict/web2 can produce some helpful results, although there are a lot of other potential data sources.