Breaking your mac with the hwprefs command
The hwprefs(1) command shows you low level information about your Mac, It’s CPU, the Memory and OS X. It also lets you control low level CPU and memory options on your mac. You can also do things like disable a CPU and disable the L1/L2/L3 CPU caches. Similar to psradm(1) in Solaris / sparc64. Be warned!
hwprefs 4.5.0 usage: hwprefs [options] parameter hwprefs [options] parameter=value OPTIONS -v verbose mode -h display this help message PROCESSOR COUNT cpu_count {1..N} number of physical processors available to MacOS X cpu_enable {1..N} enable a specific processor cpu_disable {1..N} disable a specific processor cpu_ht {true, false} processor hyperthreading PROCESSOR CACHES cpu_l1_cache {true, false} processor l1 cache cpu_l2_cache {true, false} processor l2 cache cpu_l3_cache {true, false} processor l3 cache POWER SAVING cpu_nap {true, false} processor nap INFO AND STATUS (READ ONLY) os_class displays OS class {Cheetah, Puma, Jaguar, Smeagol, Panther, Tiger} os_type displays operating system type machine_type displays machine type memory_size displays system memory cpu_type displays processor type and version cpu_freq displays processor clock frequency cpu_bus_freq displays processor bus frequency memctl_type displays memory controller type ioctl_type displays io controller type
Linux growing partitions on the fly with blockdev(1)
One thing I wished that Linux could do was to dynamically grow a partition online. This is something I was accustom to in HP-UX, AIX and even Solaris. It’s a pretty common operation to do in an enterprise environment with SAN LUNs. Lets say I have a web server running Apache on Linux. The htdocs dir sits on a dedicated SAN LUN and is slowly filling up. This server is a production box. Everything is using LVM. Before we can expand the FS, LV, or even the VG we need to grow the physical parition that the VG lives on. We expand the physical LUN size on the SAN, and fdisk the partition in Linux but linux still does not see the updated partition table size (without a reboot) – this is not good!
However, do not fear! I come across the very command to address this problem: blockdev –rereadpt /dev/sdX
Sigh… I wish I came across this command years ago!
DMT tool for Speedtouch
I’ve just come across this awesome little utility called “DMT” for Speedtouch and similar routers. Speedtouch routers have a telnet interface that let you access a fairly powerful command line interface. You can configure advanced aspects such as IDS, SNMP, etc. DMT is essentially a GUI to the telnet interface providing a complete (and awesome) overview to your Speedtouch router. Checkout the screenshot below.
DMT can be downloaded from http://www.kitz.co.uk/routers/DMTv7.htm
Home network / patchsee cables!
I’ve just moved in to my new home and I decided that I will get rid of my old long cat5e network cables and order specific length Patchsee Cat6a network cables. Whilst I don’t actually need these cables, they’re aesthetically pleasing and technically superior. I highly recommend them!
Patchsee Cat6A UTP cable
Where I work…
I recently I joined a leading e-commerce company as a UNIX system administrator.
This is where I work, and this is what I do:
Data Centre Patch Panels…
I spent Friday wiring up RJ-45 patch panels to go in to our new lovely APC racks. Usually, we pay a contractor an extortionate amount of money to do this, but there is a recession, you know! It’s a fairly simple thing to do, cut once, measure twice, learn the wiring standard, bla bla – but boy does my hand hurt after doing this all day!
Mitigating PHP attacks with noexec file system mounts
Recently, I investigated a security-compromised Apache web server. This particular web server hosted 500+ web sites and had fallen victim to script kiddies / attackers. An investigation revealed that the attackers exploitedan old Joomla CMS installation that a customer failed to update. The attackers used the Joomla vulnerability to upload and execute a binary to /var/tmp.
An analysis of this nasty executable revealed that it was apparently capable of performing SYN floods and UDP floods as well as other nasty attacks. When launched, the executable connected to a private IRC server (active-sound.de:6667) where the bot received it’s orders. The Joomla installation was subsequently brought up to date, and the binary was removed. The owner of the web server realises that these problems are typical of an unmanaged hosting business, but never the less he would like to take steps to mitigate and/or prevent this happening again.
A simple and effective solution to this problem is to remount /tmp && /var/tmp file systems with noexec,nosuid options. If it is not possible to repartition the disk, you can mount these two directories as tmpfs. This protects against any backdoors or irc DDoS bots from actually executing thus preventing the server from falling in to the hands of attackers. This protection can be mean the difference between your server being rooted and/or being turned in to a DDoS / spam drone.
Highly Available Mail Cluster – v2
It has been some time since I last blogged about my quest to build a Highly Available Mail Cluster. If you recall, the last HA Mail Cluster architecture that I designed involved four identically configured servers, spread between two data centres utilizing DNS round-robin load balancing. The Maildirs were rsynced from the ‘master node’ to the three slaves every 5 minutes. MySQL master-slave replication was used for user authentication. This worked “well enough” but it wasn’t real High Availability and it meant that, at any given time, three out of four Mail nodes were passive (idle) – wasting resources.
I decided to design a brand new platform to deliver a Highly Available Mail Cluster. The platform consists of two domU nodes in active/active configuation. Each node utilizes a DRBD block device in Primary/Primary mode. OCFS-2 is the clustered file system which sits on top of DRBD. This allows both nodes shared-concurrent access to the Maildir directories.
Both nodes run dovecot for pop3/imap and postfix for smtp. Each node has an equally weighted A/MX record for SMTP / POP3 / IMAP load balancing. The load balancing is still performed by DNS, utilizing two IP addresses in the A record. The beauty of this active/active heartbeat setup means that in the event of a server failure, the IP resource of the failed server will be taken over (via heartbeat) by the other mail server. This means there is virtually zero chance of a user hitting a stale IP address in the DNS A record. I have noticed that when users are constantly checking their inbox (pop3/imap) every 5 minutes or so Outlook caches the DNS entry indefinitely, regardless of TTL.
The above solution is working very well. I did have a couple of initial concerns regarding the stability of DRBD/OCFS-2 within a Xen domU – but I have had no problems to date. Overall the entire solution appears to be very stable.
The architecture diagram below (servers on right) shows the architecture. The full size image can also be found at: http://napta2k.googlepages.com/linode-v2.png
The drawbacks of this solution is that it does not scale past 16 nodes. The scaling limitations are due to how many cluster members can be part of Heartbeat, DRBD and OCFS-2. Thankfully I only host 400 or so mailboxes and can never see the need to scale to any more loads. One domU handles my current load just fine. The two node active/active could easily be active/passive and work just as well.
If I was going to implement a large single-data centre Highly Available Mail Cluster I would use the following architecture:
1. CARP IP address layer to distribute the load to the TCP load balancers <– probably only needed for the largest setups. You may have 16 load balancers on your front line but you definitely do not want to have 16 IP addresses in your DNS A record. CARP masquerades this.
2. TCP load balancer layer such as HAProxy to load balance the IMAP/POP3 traffic to the IMAP/POP3 farm
3. A farm of dovecot servers all sharing a _resilient_ NFS backend of Maildirs
4. A resilient NFS architecture. This could be as part of a SAN (e.g. EMC celerra) or Linux iSCSI/DRBD.
x. SMTP can be load balanced via MX.
Optimizing VBulletin for a VPS – part 1.5
I have modified my VBulletin config file and enabled the use of APC as a VBulletin datastore. Smokeping is now reporting latency of 90-95ms. Not an immediately noticeable improvement but the average load on the server is 0.00 0.00 0.00 even with 100,000 hits per day. The performance improvements should be more measurable as the load increases.
To configure APC as a VBulletin datastore I simply uncommented the following line from includes/config.php:
$config['Datastore']['class'] = 'vB_Datastore_APC';
