A.J. Clark

Solutions Architect

Google DNS vs OpenDNS – performance comparison

with one comment

Earlier today I hastily switched my router’s DNS forwarders from OpenDNS to Google DNS. Everything went OK and there aren’t any problems but I decided to perform a benchmark between OpenDNS and Google DNS for peace of mind. The results are below.

imac:~ napta2k$ ./dns_benchmark.sh
twitter.com:	 Google: ;; Query time: 18 msec		OpenDNS: ;; Query time: 16 msec
google.com:	 Google: ;; Query time: 70 msec		OpenDNS: ;; Query time: 13 msec
bbc.co.uk:	 Google: ;; Query time: 19 msec		OpenDNS: ;; Query time: 12 msec
itv.com:	 Google: ;; Query time: 20 msec		OpenDNS: ;; Query time: 16 msec
tvguide.com:	 Google: ;; Query time: 19 msec		OpenDNS: ;; Query time: 13 msec
piratebay.org:	 Google: ;; Query time: 19 msec		OpenDNS: ;; Query time: 12 msec
yahoo.com:	 Google: ;; Query time: 19 msec		OpenDNS: ;; Query time: 12 msec
wordpress.com:	 Google: ;; Query time: 19 msec		OpenDNS: ;; Query time: 13 msec
playboy.com:	 Google: ;; Query time: 184 msec		OpenDNS: ;; Query time: 15 msec

.

The test basically does a dig for each domain and records the returned Query time. The large difference in query time can be attributed to the fact that I’m physically close to the opendns.com network.

A traceroute to the OpenDNS resolvers show that my router is merely 4 hops away. A traceroute to GoogleDNS shows that my router is 20 hops away. The tests were performed from London using Level3.com as my ISP/transit. Likewise a ping to the OpenDNS resolvers show a round-trip latency of only 12ms. A ping to GoogleDNS resolvers show a round-trip latency of 20ms.

Update:

I have performed the exact same test from a server in New Jersey, USA in a nac.net data centre and OpenDNS is still far faster than GoogleDNS. Again a traceroute to OpenDNS from NJ/USA shows that the OpenDNS network is merely 5 hops away from my router. A traceroute to GoogleDNS shows that I am 10 hops away. Thus it is the network latency contributing to the overall slow query times.

napta2k@svr1:~$ ./dns_benchmark.sh
twitter.com:	 Google: ;; Query time: 10 msec		OpenDNS: ;; Query time: 1 msec
google.com:	 Google: ;; Query time: 31 msec		OpenDNS: ;; Query time: 2 msec
bbc.co.uk:	 Google: ;; Query time: 13 msec		OpenDNS: ;; Query time: 80 msec
itv.com:	 Google: ;; Query time: 15 msec		OpenDNS: ;; Query time: 1 msec
tvguide.com:	 Google: ;; Query time: 10 msec		OpenDNS: ;; Query time: 1 msec
piratebay.org:	 Google: ;; Query time: 10 msec		OpenDNS: ;; Query time: 2 msec
yahoo.com:	 Google: ;; Query time: 10 msec		OpenDNS: ;; Query time: 1 msec
wordpress.com:	 Google: ;; Query time: 10 msec		OpenDNS: ;; Query time: 2 msec
.

The results of my ad-hoc testing seem to indicate that OpenDNS has a lower latency / better placed anycast platform than GoogleDNS ?

Written by napta2k

December 4, 2009 at 4:05 pm

Posted in DNS

Tagged with , ,

Hello, NXDOMAIN!

leave a comment »

So today I moved my home router from the OpenDNS DNS servers over to the new Google public DNS resolvers. One of the reasons that I moved from OpenDNS to Google is that I got tired of the NXDOMAIN “trickery” that OpenDNS employ. I sometimes find myself doing a a fair amount of lookups with dig/nslookup and the fact that opendns will return their own IP is sometimes confusing.

My router’s DNS table originally had a single external DNS server for OpenDNS. Whilst the OpenDNS IP address is probably anycasted for performance and resilience having a single IP address is still a bit of a concern – what if the upstream route to 208.67.222.222/32 was unavailable?

:dns server route list
DNS Server       Source              Domain            Metric  Intf     State
208.67.222.222                                           10    RoutedEthoA  UP

My new solution is to load balance the Google DNS resolvers and use the OpenDNS server as a backup. The load balancing is achieved on my router by setting an identical metric. Theoretically this should make the router “flip” between DNS addresses with the same metric. I will also use OpenDNS for a backup. The Google DNS offering is quite new (released yesterday) so who knows how stable it will be.

I also have concerns about load balancing DNS IPs. You sometimes see DNS master and slaves becoming out of sync when a zone file is modified because the slave has yet to receive a NOTIFY or I/AXFR in a traditional DNS environment. Although I’m sure that the Google DNS resolvers don’t use the traditional concepts of DNS master/slaves, and instead they’re some Infrastructure 2.0 multi-master, federated, atomic entity which employ anycast.

My new DNS routing table looks like this:

:dns server route list
DNS Server       Source              Domain            Metric  Intf     State
8.8.8.8                                                  20    RoutedEthoA  UP
8.8.4.4                                                  20    RoutedEthoA  UP
208.67.222.222                                           30    RoutedEthoA  UP

As far as performance goes, I haven’t done any benchmarking between OpenDNS and Google DNS (hopefully someone will) but I am presuming that the Google offering is faster than it’s rivals, merely because it’s google.

One concern that I have about every device on your network using the router as a DNS forwarder/proxy is that, whilst you leverage things such as automatic rDNS for your home lan (e.g. DHCP client gets both an A and PTR record registered on the router) some routers experience performance problems acting as a busy DNS forwarder, and sometimes timeout / require a reboot, thus killing DNS on your network. The “safer” approach is to have your router’s DHCP server issue a set of DNS resolvers, and any modifications you make will get picked up on the next DHCP lease.

A bonus point is that if you ever travel or are setting up new equipment the Google public DNS resolver IP addresses are incredibly simple to remember: 8.8.8.8. Never again will you scramble to lookup the IP of the OpenDNS resolvers :)

Further reading:

http://code.google.com/speed/public-dns/

http://blog.opendns.com/2009/12/03/opendns-google-dns/

Written by napta2k

December 4, 2009 at 2:17 pm

Posted in DNS

Tagged with , ,

China gets horny

leave a comment »

Today I woke up to several alerts from Linode informing me that one of my VPS nodes was exceeding the Disk I/O threshold that I had set. Curiously this VPS is used as a HTTP web proxy and whilst it gets about 300-400 visitors per day (mainly china) this morning I was seeing over 800 visitors in Google Analytics.

Attempting to ssh to the server failed with timeouts although the PHP web application was still responding to requests over HTTP fine. I suspect sshd was failing to reverse-lookup my IP address in any reasonable amount of time, or perhaps IP Tables – (Note to self: Look in to why that happened). Thankfully Linode provide out of band / console access via SSH and AJax so all was not lost.

Looking at the Network rrdgraph it shows that the server was approaching 7Mbit/s of HTTP traffic and almost 50GB had been consumed today alone. Whilst the server seemed to handle the load without problem (minus ssh access) consuming 50GB+ per day would quickly max out my monthly data transfer allowance with Linode – this wasn’t acceptable. I modified the firewall to accept HTTP/HTTPS traffic from my IP only in order to investigate and the load suddenly stopped and SSH was alive again.

ehproxy.info

Initially I had suspected that some sort of automated bot was using ehproxy.info to do automated scans and attacks but a closer inspection of the traffic showed an even number of distributed IPs (all from China – as Google Analytics confirms) all clicking various porn sites. I guess everyone in China was feeling horny this afternoon!

most visited sites

Further analysis of the access.log shows that the server (Linode XenU VPS with 720MB of ram) was handling 62 hits sec (2428863/39600) and lighttpd was dealing with the load no problem. Pretty good considering this is a pure PHP application utilising php-cgi.

For the record, the top five IP addresses were:

Hits : IP address

13872 :222.242.182.169
14394 :123.152.129.107
16422 :123.11.41.255
18110 :113.65.146.126
26595 :124.134.24.218

Written by napta2k

November 1, 2009 at 5:06 pm

Posted in Linux, Xen, lighttpd

Show limits of a running process in Linux

leave a comment »

A rather simple but often asked question was put forward to me today: How can I see the maximum amount of file descriptors my running process can open? (without killing the process!)

Typically one would say ‘check ulimit -n’ but lets say that a thread-driven or event-driven application like varnish or lighttpd is configured with an arbitrary amount of open file descriptors and you want to verify that they have taken effect before the application crashes.

A simple way to check this (atleast on Linux 2.6.26-1 or later) is to run:

svr1:~# awk ‘/Max open files/{ print $4}’ /proc/$(pgrep -n apache)/limits
1024
svr1:~# awk '/Max open files/{ print $4}' /proc/$(pgrep -n lighttpd)/limits

1024

As you can see the above command returned the value of max open files for the running process. This means you can be sure that your lighttpd or varnish application will not suddenly die after being starved of file descriptors!

I have included the entire output of the limits table for the lighttpd process for completeness:

svr1:~# cat /proc/$(pgrep -n lighttpd)/limits
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            ms
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             5824                 5824                 processes
Max open files            1024                 1024                 files
Max locked memory         32768                32768                bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       5824                 5824                 signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us

Written by napta2k

September 29, 2009 at 2:23 pm

Posted in Linux, lighttpd

Error: Device 0 (vif) could not be connected. Hotplug scripts not working

leave a comment »

Are you running Xen / “xm create” and you get this error?

Try this (RHEL/CentOS): service haldaemon start

- and have a nice day!

Written by napta2k

September 25, 2009 at 7:43 pm

Posted in Linux, Red Hat, Xen

Breaking your mac with the hwprefs command

leave a comment »

The hwprefs(1) command shows you low level information about your Mac, It’s CPU, the Memory and OS X. It also lets you control low level CPU and memory options on your mac. You can also do things like disable a CPU and disable the L1/L2/L3 CPU caches. Similar to psradm(1) in Solaris / sparc64. Be warned!

hwprefs 4.5.0
usage: hwprefs [options] parameter
hwprefs [options] parameter=value
OPTIONS
-v verbose mode
-h display this help message
PROCESSOR COUNT
cpu_count {1..N} number of physical processors available to MacOS X
cpu_enable {1..N} enable a specific processor
cpu_disable {1..N} disable a specific processor
cpu_ht {true, false} processor hyperthreading
PROCESSOR CACHES
cpu_l1_cache {true, false} processor l1 cache
cpu_l2_cache {true, false} processor l2 cache
cpu_l3_cache {true, false} processor l3 cache
POWER SAVING
cpu_nap {true, false} processor nap
INFO AND STATUS (READ ONLY)
os_class displays OS class {Cheetah, Puma, Jaguar, Smeagol, Panther, Tiger}
os_type displays operating system type
machine_type displays machine type
memory_size displays system memory
cpu_type displays processor type and version
cpu_freq displays processor clock frequency
cpu_bus_freq displays processor bus frequency
memctl_type displays memory controller type
ioctl_type displays io controller type
hwprefs 4.5.0

usage: hwprefs [options] parameter      

       hwprefs [options] parameter=value

OPTIONS

 -v verbose mode

 -h display this help message

PROCESSOR COUNT

 cpu_count {1..N} number of physical processors available to MacOS X

 cpu_enable {1..N} enable a specific processor

 cpu_disable {1..N} disable a specific processor

 cpu_ht {true, false} processor hyperthreading

PROCESSOR CACHES

 cpu_l1_cache {true, false} processor l1 cache

 cpu_l2_cache {true, false} processor l2 cache

 cpu_l3_cache {true, false} processor l3 cache

POWER SAVING

 cpu_nap {true, false} processor nap

INFO AND STATUS (READ ONLY)

 os_class displays OS class {Cheetah, Puma, Jaguar, Smeagol, Panther, Tiger}

 os_type displays operating system type

 machine_type displays machine type

 memory_size displays system memory

 cpu_type displays processor type and version

 cpu_freq displays processor clock frequency

 cpu_bus_freq displays processor bus frequency

 memctl_type displays memory controller type

 ioctl_type displays io controller type

Written by napta2k

June 13, 2009 at 10:47 pm

Posted in OS X

Linux growing partitions on the fly with blockdev(1)

leave a comment »

One thing I wished that Linux could do was to dynamically grow a partition online. This is something I was accustom to in HP-UX, AIX and even Solaris. It’s a pretty common operation to do in an enterprise environment with SAN LUNs. Lets say I have a web server running Apache on Linux. The htdocs dir sits on a dedicated SAN LUN and is slowly filling up. This server is a production box. Everything is using LVM. Before we can expand the FS, LV, or even the VG we need to grow the physical parition that the VG lives on. We expand the physical LUN size on the SAN, and fdisk the partition in Linux but linux still does not see the updated partition table size (without a reboot) – this is not good!

 

However, do not fear! I come across the very command to address this problem: blockdev –rereadpt /dev/sdX

 

Sigh… I wish I came across this command years ago!

Written by napta2k

May 24, 2009 at 1:55 pm

Posted in Uncategorized

DMT tool for Speedtouch

leave a comment »

I’ve just come across this awesome little utility called “DMT” for Speedtouch and similar routers. Speedtouch routers have a telnet interface that let you access a fairly powerful command line interface. You can configure advanced aspects such as IDS, SNMP, etc. DMT is essentially a GUI to the telnet interface providing a complete (and awesome) overview to your Speedtouch router. Checkout the screenshot below.

DMT can be downloaded from http://www.kitz.co.uk/routers/DMTv7.htm

dmt20090522_1450

Written by napta2k

May 22, 2009 at 1:58 pm

Posted in Uncategorized

Home network / patchsee cables!

leave a comment »

I’ve just moved in to my new home and I decided that I will get rid of my old long cat5e network cables and order specific length Patchsee Cat6a network cables. Whilst I don’t actually need these cables, they’re aesthetically pleasing and technically superior. I highly recommend them!

 

 

Patchsee Cat6A UTP cable

Patchsee Cat6A UTP cable

Written by napta2k

April 16, 2009 at 7:07 pm

Posted in Uncategorized

The golden quad-core

leave a comment »

A screenshot from a console session to a Rackable server:

mac-address-weirdness

Written by napta2k

April 7, 2009 at 6:41 pm

Posted in Uncategorized