A Little Bit Faster Now

The DSL link was upgraded from 1.5Mb/s down and 256kb/s up to 3Mb/s and 512kb/s respectively this morning. Which reminds me that getting the most out of the link requires some work. Daniel Hartmeier has a nice write-up on prioritizing ACKs which is good idea on PPPoE links so that you can upload and download simultaneously without killing your throughput one way or the other.

PPPoE especially seems to have issues with empty ACKs getting caught in buffers and having to wait 1 second or more to be fed through and out. The following user PPP setting in FreeBSD is of particular interest.

set ifqueue packets




Set the maximum number of packets that ppp will read from the
tunnel interface while data cannot be sent to any of the available links. This queue limit is necessary to flow control outgoing data as the tunnel interface is likely to be far faster than
the combined links available to ppp.





If packets is set to a value less than the number of links, ppp
will read up to that value regardless. This prevents any possible latency problems.





The default value for packets is “30”.

If we’re uploading a lot of data, those 30 default packets will tend to be the maximum MTU size (generally 1492 for PPPoE). That’s 44KBytes of data waiting around to be transmitted, or 358kbits. If the link is only pushing 256kbits each second, new items in the queue have to wait nearly a second and a half to get out. If your TCP ACKs for downloads start getting delayed by that long, the throughput on that connection is going to suffer and VoIP or gaming is simply out of the question.

With faster links, this default is somewhat less problematic. Our new 512kb/s link still has a buffer of 700ms or so. Still more than enough to confuse the heck out of the in-flight packet management in TCP when that buffer wait starts fluctuating wildly.

I suspect if ifqueue was set very small and some sort of weighted fair queuing was used to manage its input, that total performance would be better. And it would avoid having to guess the optimal upstream bandwidth the way you do with the suggested approach with pf.