Today between 11am-12pm Hong Kong time, Pacnet‘s routing tables got completely screwed up. How, I don’t know, but I was getting traceroute results like this:
C:\Documents and Settings\Ali>tracert news.bbc.co.uk
Tracing route to newswww.bbc.net.uk [212.58.226.75]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms 192.168.1.1 2 20 ms 17 ms 19 ms crsall.pacific.net.hk [202.64.10.27] 3 198 ms 242 ms 224 ms v152.tmhc1.pacific.net.hk [202.64.4.1] 4 * * * Request timed out. C:\Documents and Settings\Ali>tracert www.ebrahim.org
Tracing route to ebrahim.org [216.92.73.127]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms 192.168.1.1 2 23 ms 20 ms 18 ms crsall.pacific.net.hk [202.64.10.27] 3 19 ms 17 ms 19 ms v152.tmhc1.pacific.net.hk [202.64.4.1] 4 * * * Request timed out. C:\Documents and Settings\Ali>tracert blog.ebrahim.org
Tracing route to blog.ebrahim.org [66.39.91.220]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms 192.168.1.1 2 19 ms 18 ms 18 ms crsall.pacific.net.hk [202.64.10.27] 3 20 ms 20 ms 20 ms g2-1.tmhr7601.pacific.net.hk [202.64.4.150] 4 20 ms 19 ms 20 ms Gi1-7.gw1.hkg4.asianetcom.net [203.192.153.225] 5 18 ms 19 ms 23 ms gi1-0-0.cr3.hkg3.asianetcom.net [202.147.16.201] 6 78 ms 81 ms 79 ms po13-0-1.cr1.nrt1.asianetcom.net [202.147.0.61] 7 189 ms 192 ms 198 ms po1-0.gw1.sjc1.asianetcom.net [202.147.50.133] 8 206 ms 204 ms 204 ms gige-g5-10.core1.sjc2.he.net [72.52.99.5] 9 247 ms 247 ms 246 ms 10gigabitethernet1-1.core1.chi1.he.net [72.52.92.74] 10 249 ms 250 ms 251 ms a1-0-3.br1.snjsca.us.lightning.net [216.66.3.42] 11 * * * Request timed out. 12 254 ms 254 ms 253 ms blog.ebrahim.org [66.39.91.220] Trace complete. What's the real kicker? That blog.ebrahim.org and www.ebrahim.org are hosted on the same machine, but that I can get to one and not the other. Judging by the vague similarity between the IP addresses of www.ebrahim.org and news.bbc.co.uk (both are 21x.x.x.x, which is very vague indeed), I figure that Pacnet has their routing tables all messed up.
At around 11:30 or 11:45am I call up their support to report the problem, and am asked to send them a traceroute report. That makes sense, and I do that. I ask when I can expect a resolution and the drone on the other end tells me that since today is a Sunday, nobody at their NOC is at work, and I can expect a resolution on Monday or Tuesday, up to a full 2 days later.
That’s really appalling support, Pacnet. If you have routing issues that are blocking off entire swaths of the internet, you need to fix this yesterday, not two days later. What kind of NOC goes on holiday on a Sunday? Does the internet take a rest day too?
In case anybody at Pacnet is keeping score, my ticket number is TEC3511 and it’s now 7:15pm and it looks like nothing has been done.