A Little Bit Faster Now

The DSL link was upgraded from 1.5Mb/s down and 256kb/s up to 3Mb/s and 512kb/s respectively this morning. Which reminds me that getting the most out of the link requires some work. Daniel Hartmeier has a nice write-up on prioritizing ACKs which is good idea on PPPoE links so that you can upload and download simultaneously without killing your throughput one way or the other.

PPPoE especially seems to have issues with empty ACKs getting caught in buffers and having to wait 1 second or more to be fed through and out. The following user PPP setting in FreeBSD is of particular interest.

set ifqueue packets




Set the maximum number of packets that ppp will read from the
tunnel interface while data cannot be sent to any of the available links. This queue limit is necessary to flow control outgoing data as the tunnel interface is likely to be far faster than
the combined links available to ppp.





If packets is set to a value less than the number of links, ppp
will read up to that value regardless. This prevents any possible latency problems.





The default value for packets is “30”.

If we’re uploading a lot of data, those 30 default packets will tend to be the maximum MTU size (generally 1492 for PPPoE). That’s 44KBytes of data waiting around to be transmitted, or 358kbits. If the link is only pushing 256kbits each second, new items in the queue have to wait nearly a second and a half to get out. If your TCP ACKs for downloads start getting delayed by that long, the throughput on that connection is going to suffer and VoIP or gaming is simply out of the question.

With faster links, this default is somewhat less problematic. Our new 512kb/s link still has a buffer of 700ms or so. Still more than enough to confuse the heck out of the in-flight packet management in TCP when that buffer wait starts fluctuating wildly.

I suspect if ifqueue was set very small and some sort of weighted fair queuing was used to manage its input, that total performance would be better. And it would avoid having to guess the optimal upstream bandwidth the way you do with the suggested approach with pf.

Universal Build of PostgreSQL 8.1.3

Traditionally I’ve used DarwinPorts for managing the packages installed on my Mac laptop. Sadly, there has been some falling out between the OpenDarwin folks and Apple recently. For this and other reasons having DarwinPorts build Mac OS X-friendly Intel binaries cleanly (let alone universal binaries) seems highly unlikely.

In order to produce things that’ll work on both PowerPC Macs and ICBMs (Intel-CPU based Macs, what an acronym) you have to build it yourself or find someone else who has taken the time to do it.

The Mac Python crew have been working for a couple months now on a universal Python 2.4.2 install that passes its own regression testing (unlike the one that Apple ships). They have a copy of the installer .dmg available online. (There is also a universal Ruby online if you swing that way.)

One of the problems I’ve encountered, after using these builds, is that the universal Python really wants its C-based modules compiled as universal builds also. It’s not such a big deal when the modules are standalone (and pay attention to the CFLAGS that the Python build was made with). It is more problematic when they want to link against other things, which then must also be built universal.

In today’s adventure, I wanted to build the psycopg module that is a way to access PostgreSQL databases from inside Python. This meant making a universal build of PostgreSQL. Marc Liyanage has some universal builds available, but they were for 8.1.2 rather than 8.1.3. His ViewCVS is install is broken right now, but I was able to get the Perl snippet from the Google cache to get the build to succeed.

PostgreSQL actually builds remarkably easy aside from an oft-repeated bit where it takes the myriad object files (.o) from each smaller component and makes them into one larger object file. Apparently ld doesn’t know how to consolidate “fat” object files. To workaround this you have to run ld once for each architecture (ppc, i386) then lipo the resultant files together. Marc’s one-liner takes care of this change in the various makefiles.

find . -name Makefile -print -exec perl -p -i.backup -e 's/\Q$(LD) $(LDREL) $(LDOUT)\E (\S+) (.+)/\$(LD) -arch ppc \$(LDREL) \$(LDOUT) $1.ppc $2; \$(LD) -arch i386 \$(LDREL) \$(LDOUT) $1.i386 $2; lipo -create -output $1 $1.ppc $1.i386/' {} \;

Psycopg ignores CFLAGS given to it during configure and took a slight bit of monkeying around to use the magic flags that are in the Apple documentation to build. Other than that it went smoothly though.

I stuck the PostgreSQL build into /Library/Frameworks/PostgreSQL.framework for good measure. I’m not sure if that’s a good or bad idea, but it does keep it from getting stomped on by native builds done in OpenDarwin or Fink or whatever.

It’s about 8MB all wrapped up so I’m not going to post it here, but if people really want it maybe I can work something out. That’s all for now…

USB Audio Out Broken on Intel Macs

Yes, the joys of having a Rev. A hardware product from Apple are boundless. My new MacBook Pro seems to have a problem dealing with USB audio output devices. According to a reply I got on Apple’s discussion boards this affects Intel-based iMacs as well.

Thanks to the fact that Apple won’t open the xnu (Mac OS X kernel) sources for the Intel systems, I can’t effectively chase down the issue myself (potentially saving some beleaguered Apple engineer some work rounding up a suitable test rig). So I’ve resorted to filing a bug with the Apple’s developer feedback interface: 4460973 – USB Audio Output Fails on Intel Macs. I’ve never used this before and my previous experience with Apple’s bugfix process was lackluster.

1. plug in a USB audio output device


2. select it for output in System Preferences


3. click Play in iTunes

Feb 25 21:57:09 macbook kernel [0]: USBF: 89320.749 IOUSBPipe[0x4244600]:ClosePipe for address 6, ep 1 had a retain count > 1. Leaking a pipe

I always figured there would be some sort of minor issue like this, being Rev. A hardware and all. On the whole the machine is great, but I imagine this would be a serious bit of sadness for DJ types that weren’t just using the built-in outputs. USB audio output devices work if, and only if, the device was plugged in at boot and the machine hasn’t been in “sleep” mode between then and now. So just reboot, over and over and over again! I’d just as soon leave that fun usability experience to the folks with that other operating system.

It’ll be interesting to see how quick Apple can get a fix for this. Since it affects iMac users as well it’s most likely a problem with the USB controller or the driver for it. However, it’s hard to say for sure without digging a lot deeper, and dig_deeper() is returning ENOXNUSRC.

Universal Build of SSHKeychain

I recently received my new MacBook Pro and have been slowly working through getting all the programs I want running nicely. Native Intel code screams compared to the PowerBook I had been using previously, but sadly not all programs have been rebuilt for both platforms yet.

SSHKeychain is an extraordinarily useful program, but is missing a “Universal” (Intel + PowerPC machine code) build. Luckily it is open source, so I was able to simply build it myself. You can get a copy with this link. It is subject to a BSD-style license which is included in the zip file. If you’re nervous about .zip files after the recent virus shenanigans there is also a detached signature.

Many thanks to Bart Matthaei who originally developed this indispensable tool.