Saint Aardvark's Journal: OpenWRT/VPN/NTPD

I've been setting up OpenVPN on my wife's iBook, using 3.0-RC2 of Tunnelblick. It works well, but I did come across one bit of weirdness.

I'm using OpenVPN in bridging mode. The network looks like this:

iBook <-> WRT54G <-> Home Network <-> Firewall <-> Internet

When OpenVPN bridging is going on, the iBook appears to be sitting on the Home Network. During testing, I was able to ping the Firewall box and other boxes on the Home Network, but I was unable to connect to websites on the Internet, and pinging Internet hosts got me this error:

ping: sendto: No buffer space available
ping: wrote www.google.ca 64 chars, ret=-1

I've run into this problem before, but the last time it was because one end of the tunnel went down -- not the case now.

I tried looking at netstat -m but it all looked good:

98 mbufs in use:
94 mbufs allocated to data
3 mbufs allocated to socket names and addresses
1 mbufs allocated to Appletalk data blocks
145/368 mbuf clusters in use
760 Kbytes allocated to network (41% in use)
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines

...which pretty much matches what I've seen when I've had this message before in other situations. netstat -s was a little more interesting:

icmp:
266 calls to icmp_error
0 errors not generated 'cuz old message was icmp
Output histogram:
echo reply: 2
destination unreachable: 266
0 messages with bad code fields
0 messages < minimum length
0 bad checksums
0 messages with bad length
0 multicast echo requests ignored
0 multicast timestamp requests ignored
Input histogram:
echo reply: 865
destination unreachable: 284
routing redirect: 18
echo: 2
2 message responses generated
ICMP address mask responses are disabled

"Destination unreachable"? WTF? I had a look at the interfaces, and saw three tap instances. Not only that: tap1 was the one being used by Tunnelblick, but tap0 was the default gateway. That can't be right, right? Right. Turned out there was another, older instance of OpenVPN running that should've been killed long ago. Kill that, kill Tunnelblick, restart Tunnelblick, and all is well.

I'm hoping I won't have this crop up again; it's not in my wife's nature to start looking for rogue tap interfaces screwing up the routing tables if her Internet connection goes down. :-)

One other problem I had with the WRT54G was bridging and setting the time. See, OpenVPN starts at boot on this thing, and it needs a tap interface. Since we're doing bridging, it needs to be bridged to the outside interface -- vlan1 on OpenWRT, the Linux firmware I'm using. In order to make that work, I get the firewall script to create the bridge right before it runs all the firewall rules. That happens right before OpenVPN starts, which happens right before OpenNTPD runs, which happens right before the boot scripts finish and we're open for business. Firewall, then OpenVPN, then OpenNTPD. Got it?

Setting the time is important because otherwise OpenVPN will complain that the iBook is connecting using an SSL certificate that's not valid yet, and refuses to connect. Well, no problem -- ntpd -s takes care of that, right? Wrong: every time I checked the date, the time was 1999. (Yeah, I could've tried to set the hardware clock to something more reasonable, but that's a stupid hack.)

ntpd -s was running but not setting the time. tcpdump on my NTP server showed that there were no NTP requests coming from the WRT after it booted. Yet I could kill ntpd, start it up again, and it would set the time right away. I tried ntpclient, but it behaved in exactly the same way.

In the end, I couldn't find out what was going on. I suspect it's a problem related to the bridge I set up -- ntpd binds to vlan1, then for some reason things stop working once the bridge is set up. I can't be sure without a serial port, though -- still haven't figured out logging on this thing -- so I used a slightly less awful hack: running ntpclient to set the time just before bridging starts, then running openntpd after as usual. It just dodges the problem, rather than fixing it, but it works.