Article posted on Jun 24

Hampr Mobile updated

• Posted by Ryan Finnie on June 24, 2010, 8:03 pm

Hampr Mobile, the Hampr interface designed for WebKit-based mobile phone browsers (iPhone, Android, WebOS) has received a facelift today. Text has been updated to look better on the higher resolution of the new iPhone 4 released today (while looking just as well as it did before on other phones' browsers), and the theme has been prettied up, with a new brushed metal theme. Some new iPhone-specific options have been added. If you add Hampr Mobile to your home screen (press the "+" sign at the bottom, then select "Add to Home Screen"), a new, high-resolution icon is added that is iPhone 4 compatible. Give it a try!

• Leave Comment
• Posted in Hampr

Article posted on Jun 7

HOWTO: Delete a 3ware DCB

• Posted by Ryan Finnie on June 7, 2010, 1:51 pm

I have a large enough infrastructure at work where we're often (once or twice per month) moving disks around, replacing failed disks on 3ware controllers, etc. One thing that often shows up is issues when an unwiped disk previously used in a 3ware controller is put into another 3ware controller. First, let's step back:

Update (2010-06-08): I added/updated some information based on what I've found analyzing a handful of DCBs in the last few days.

3ware DCB analysis

LSI 3ware (née AMCC 3ware, née 3ware) uses a proprietary disk layout that stores RAID information on the disks themselves. This has the advantage of being able to replace/upgrade a controller without having to reconfigure it. However, if you move a disk that was previously part of another array into a new machine without wiping it, it shows up to the new array as a failed existing array.

3ware stores this information on a disk in a section called the Disk Control Block (DCB). There are two versions of the DCB which I refer to as Old style and New style. On both Old and New, the DCB is stored at the end of the disk. I have discovered that the DCB appears to be 1024 LBAs long (1024 * 512 bytes per sector = 512KiB). Additionally, the Old style includes a copy of the DCB as the first 1024 LBAs of the disk. The actual disk data starts at LBA 0 on New style and LBA 1024 on Old style.

The Old and New style DCBs' 512KiB blocks are laid out differently, and I think I have found magic strings that can differentiate between them. On Old style, there is the string \x03\x00\xe9\x07 2044 bytes into the DCB, while New style has the string '3WareDCB' 2056 bytes into the DCB.

All 9000 series controllers I've seen so far seem to use New style, and until recently, all 8000 series controllers I had seen had Old style. I thought it was an 8000-versus-9000 thing, but I recently came across an 8006-2LP 2-disk RAID1 array that had a New style DCB. Updated firmware perhaps? Even odder about this array was that on all the New style DCBs I had seen until then, the actual disk data began at LBA 0, that is the beginning of the disk. However, on these disks, the disk data began at LBA 8. LBA 0-7 (4 KiB) contained something that sort of looked like a DCB (serial numbers are in there), but is much too short to be a proper DCB. This suggests that, at least with New style DCBs, the beginning data offset is stored somewhere within the DCB, but I have not yet been able to find it.

It should be noted that the DCBs on 8000 and 9000 series controllers are not compatible with each other. Plugging a disk previously used in an 8000 series controller (and not wiped) into a 9000 series controller will produce the dreaded UNCONV-DCB error. The disk is still usable as a (new) member of an existing array on the 9000, but it appears you cannot get rid of the UNCONV-DCB error until you take the disk out and manually wipe it. 9000 series are backwards and forwards compatible with each other, so you could take the disks from, say, a 9650SE and put them in a 9550SX or vice versa.

The previous paragraph's information comes from 3ware's docs and my personal experience, but that recent discovery of an 8006-2LP with a New style DCB throws a wrench into things. It could be that 3ware is actually referring to Old and New style DCBs when they say 8000 and 9000. If that's true, I may be able to put those New style 8006-2LP disks into a 9000 series controller. I'll have to try that sometime.

Wiping the DCB

So, the important part to take away is if you want to reuse a disk that was previously in a 3ware controller on another 3ware controller as if it were a blank disk, it is best to wipe the disk's DCB areas first, to make sure the new controller doesn't recognize it. I'm currently writing a utility that can search a disk for DCBs, tell the difference between Old and New style DCBs, dump them to files, wipe them, etc. But for now, here's a short Linux shell script for wiping the DCB areas from a disk:

DISK=sdz
LBAS=$(cat /sys/block/$DISK/size)
dd if=/dev/zero of=/dev/$DISK bs=512 count=1024
dd if=/dev/zero of=/dev/$DISK bs=512 seek=$(($LBAS-1024)) count=1024

Replace "sdz" with the device name (triple check; I'm not responsible for you hosing your boot drive). You can skip the first "dd" command if you are sure that the disk came from a disk with a New style DCB, but if you're "wiping" the disk with the intent of reformatting, it doesn't hurt to erase both the beginning and ending 1024 LBAs.

• 1 Comment
• Posted in Uncategorized

Article posted on Apr 30

IPv6: Living in the future, etc, etc

• Posted by Ryan Finnie on April 30, 2010, 7:48 pm

Well, it's 2010, and I'm finally living in The Future. I now have an ISP that is offering native IPv6, my colo provider. I switched over my irssi connections to make sure all IPv6-capable IRC networks are connecting via IPv6. (OFTC, Foonetic, Coldfront, Freenode -- sadly, Slashnet is the only one without an IPv6 server.) I have also added AAAA records to many of my hosted domains, such as finnie.org, finnix.org, x11r5.com, velociraptors.info and hampr.com. Also, I can reasonably assume I am running the world's only IPv6 TCPMUX server.

While I do not have native IPv6 at home, I am using Hurricane Electric's Tunnel Broker service. I'm not going to go into details on how to set this up, but I do want to stress the importance of firewalling IPv6. In the IPv4 world, NAT is used as a security crutch. In "The Future", when everything is IPv6, NAT will be irrelevant, but because of that, firewalling is that much more important. If you use a IPv6 tunneling service, be sure your internal LAN (which then becomes an external LAN) is properly firewalled.

In Linux, this requires the use of ip6tables. I personally use a simple setup, allowing outbound traffic, inbound ICMP, inbound SSH and a few select inbound services to individual machines. I've included my ip6tables config below.

# Set a default DROP policy.  Note that this only affects IPv6 traffic,
# it does not affect the regular iptables FORWARD policy.
ip6tables -P FORWARD DROP
# Allow any outbound traffic from your local LAN (2001:470:1f05:22e::/64).
# Replace "hetunnel" with your tunnel/outbound interface (or leave it off,
# though it helps prevent possible spoofing).
ip6tables -A FORWARD -s 2001:470:1f05:22e::/64 -o hetunnel -j ACCEPT
# Allow any established inbound or outbound traffic.
ip6tables -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
# Allow ICMP inbound.  "ipv6-icmp" is required for ip6tables here.
ip6tables -A FORWARD -p ipv6-icmp -j ACCEPT
# Allow SSH inbound to any host.
ip6tables -A FORWARD -p tcp -m tcp --dport 22 -m state --state NEW -j ACCEPT
# Allow µTorrent inbound tcp/udp, to my Vista machine in this case.
ip6tables -A FORWARD -p tcp -d 2001:470:1f05:22e:24c3:ff01:e72a:3487 \
  --dport 30173 -m state --state NEW -j ACCEPT
ip6tables -A FORWARD -p udp -d 2001:470:1f05:22e:24c3:ff01:e72a:3487 \
  --dport 30173 -m state --state NEW -j ACCEPT

• Leave Comment
• Posted in Uncategorized

Article posted on Mar 7

External temperature monitoring with Linux

• Posted by Ryan Finnie on March 7, 2010, 3:36 pm

At work, we moved into a new office last week. Part of that move involved moving the office network infrastructure from a massive server and work room into a small server closet. We're having some issues with cooling, so I decided to get some sort of ambient temperature monitoring going while we work on fixing the issue.

I found a few DIY articles (such as here and here) that use the Dallas Semiconductor 1-Wire bus. 1-Wire is a low-speed parallel communications bus where the entire bus is powered and controlled by a single master interface. The master is usually a serial device connected to a computer. The rest of the devices can either be daisy-chained, or wired in a star topology, much the same as home phone wiring can be split from the telco demarc to multiple phones. (Despite the name, two wires are needed for a 1-Wire bus, one carries data and power, the other is ground.)

They looked nice, but I didn't want to go the DIY route. Eventually I found iButtonLink, who makes and sells 1-Wire products. I bought the LinkUSBi master and a few T-Sense temperature probes. (The LinkUSB is a few dollars cheaper than the LinkUSBi, but the LinkUSBi also contains an ID chip, so you can verify operation of the bus even if no other devices are connected.) The iButtonLink products use RJ45 ports, so you can use Cat5 network cable to connect them together. The T-Sense probe has RJ45 ports on both ends, so you can daisy chain them if you like. I haven't done so, but if you wanted to do star topology instead, you could buy a 4-port biscuit block with 4 RJ45 ports, and then wire all of their pin 4s together and all of their pin 5s together. Then simply plug the master into one port (doesn't matter which, remember this is a parallel bus), and the sensors into the rest.

The products arrived Thursday, and I plugged the master into a server running Debian 5.0 Lenny. The LinkUSBi is a serial device with an internal FTDI FT232R USB UART, which was recognized by the ftdi_sio usb-serial module and set up as /dev/ttyUSB0. In case you have other ttyUSB devices on your machine, I devised this udev rule for more permanent device naming:

SUBSYSTEM=="tty", ATTRS{serial}=="?*", SYMLINK+="char/by-id/tty-$attr{serial}"

In this case it gives me /dev/char/by-id/tty-A800bZvc, but for the sake of simplicity I will continue to refer to the device as /dev/ttyUSB0 for the rest of this guide.

digitemp

Now, to read the temperatures. You'll want the digitemp package, which is pre-packaged in Debian and probably most other Linux distributions. Once it's installed, you'll need to know what program to use. Several are available for different DS masters, but in the LinkUSBi's case, you'll want digitemp_DS9097U. To begin, verify the bus is working correctly by walking it:

# digitemp_DS9097U -w -s /dev/ttyUSB0
DigiTemp v3.5.0 Copyright 1996-2007 by Brian C. Lane
GNU Public License v2.0 - http://www.digitemp.com
Turning off all DS2409 Couplers
...
Devices on the Main LAN
28D1483C0200002F : DS18B20 Temperature Sensor
28E9393C020000C3 : DS18B20 Temperature Sensor
010EBED512000046 : DS2401/DS1990A Serial Number iButton

This shows the two DS18B20 temperature sensors (T-Sense probes), as well as the DS2401 embedded in my LinkUSBi. (The DS2401 literally does nothing but return a serial number. Still, as mentioned before it's useful to have to verify the bus is working correctly even if no other devices are plugged into it.)

Next you'll want to create a config file. I chose to store it in /etc/digitemp.conf.

# digitemp_DS9097U -i -c /etc/digitemp.conf -s /dev/ttyUSB0
DigiTemp v3.5.0 Copyright 1996-2007 by Brian C. Lane
GNU Public License v2.0 - http://www.digitemp.com
Turning off all DS2409 Couplers
...
Searching the 1-Wire LAN
28D1483C0200002F : DS18B20 Temperature Sensor
28E9393C020000C3 : DS18B20 Temperature Sensor
ROM #0 : 28D1483C0200002F
ROM #1 : 28E9393C020000C3
Wrote /etc/digitemp.conf

You will be left with a file called /etc/digitemp.conf that looks something like this:

TTY /dev/ttyUSB0
READ_TIME 1000
LOG_TYPE 1
LOG_FORMAT "%b %d %H:%M:%S Sensor %s C: %.2C F: %.2F"
CNT_FORMAT "%b %d %H:%M:%S Sensor %s #%n %C"
HUM_FORMAT "%b %d %H:%M:%S Sensor %s C: %.2C F: %.2F H: %h%%"
SENSORS 2
ROM 0 0x28 0xD1 0x48 0x3C 0x02 0x00 0x00 0x2F
ROM 1 0x28 0xE9 0x39 0x3C 0x02 0x00 0x00 0xC3 

You can rearrange the ROM mappings as you'd like. 1-Wire refers to devices by their 64-bit IDs; the numeric mappings are for digitemp's benefit. In my case, 28D1483C0200002F is the probe I'd like to use, so #0 is fine.

Now, let's see what's being returned:

# digitemp_DS9097U -q -c /etc/digitemp.conf -a
Mar 04 22:22:43 Sensor 0 C: 25.38 F: 77.67
Mar 04 22:22:44 Sensor 1 C: 27.56 F: 81.61

Great, works fine. We'll soon need the data in a machine-readable format, so here's how to do that:

# digitemp_DS9097U -q -c /etc/digitemp.conf -o 3 -a
0	77.79	81.50

That output is tab-delimited, the first column being elapsed time (digitemp can pull data multiple times, but we won't be going into that so it will always be 0 here), and the rest of the columns are the probe values in order. "-o 3" is Fahrenheit; use "-o 2" for Celsius. In this case I only want the result of the first probe, so I can save some time by specifying a specific probe with "-t 0":

# digitemp_DS9097U -q -c /etc/digitemp.conf -o 3 -t 0
0	77.90

Read the rest of this entry »

• 3 Comments
• Posted in Uncategorized

Article posted on Feb 18

Cisco PIX DNS fixup in Linux?

• Posted by Ryan Finnie on February 18, 2010, 7:12 pm

At work we have a Cisco PIX firewall for the office. It's decent (if a bit eccentric; that is, hard to configure), but occasionally I go through a thought exercise to see how this firewall could be replaced with a Linux firewall. Most of the functionality is easy in Linux (NAT, ACLs, VPNs, etc), but one thing I get hung up on is DNS fixup. Fixup is a monitoring service much like nf_conntrack/nf_nat in Linux, and in DNS fixup's case can rewrite responses depending on the context. Here's an explanation:

The players:
- Mallory is the PIX firewall, with the 10.0.0.0 network inside and the 9.9.9.0 network outside. (Despite conventional naming examples, Mallory is not malicious here, but otherwise has the same attributes.)
- Alice is the DNS server, 10.0.0.2 inside, 9.9.9.2 outside. Alice knows only about internal IPs in her DNS database.
- Bob is some server, 10.0.0.3 inside, 9.9.9.3 outside. Bob is listed with Alice as bob.corp.example.com, 10.0.0.3.
- Charlie is a client on the outside network.
- Dave is a client on the inside network.

Now, say Charlie (outside) queries bob.corp.example.com via Alice's external IP. Alice will respond with 10.0.0.3. Mallory intercepts the response, knows that Bob is 10.0.0.3 on the inside and 9.9.9.3 on the outside, so she rewrites the response as 9.9.9.3 and gives it to Charlie.

It also works in the opposite direction. Say www.example.com is a web server served by Bob, and DNS is hosted by an outside DNS provider which obviously returns 9.9.9.3 for www.example.com. Now say Dave (inside) queries www.example.com via Alice. Alice doesn't know about www.example.com, so she goes out to the Internet (through Mallory) to find it. The outside DNS responds with 9.9.9.3. Again, Mallory knows about Bob's mapping and will rewrite the response to 10.0.0.3 to Alice, which then gives the final answer to Dave.

As far as I know, there is nothing in Linux to facilitate this. Yes, I know about split-horizon DNS, but it's a pain to maintain multiple zone copies, and Alice's DNS service would have to be moved to Mallory directly. The PIX does this all automatically for you (if you want; of course it can be disabled).

(Please, prove me wrong.)

• Leave Comment
• Posted in Uncategorized

Article posted on Feb 13

in.tcpmuxd: A secure, RFC compliant TCPMUX server

• Posted by Ryan Finnie on February 13, 2010, 3:41 am

Yesterday, on IRC, neale asked if it was wise to run a TCP service on port 1. sneakums replied it was not, since it was a registered service, "tcpmux". However, nobody immediately knew what "tcpmux" was; Wikipedia provided the answer.

TCPMUX is an ancient, horrible protocol. You connect to a TCPMUX server on port 1, then tell it which TCP service you actually wanted, and it forwards locally for you. Obviously fraught with security problems on the modern Internet. Nonetheless, I immediately wanted to write a TCPMUX server.

I started out by coding to the description in the Wikipedia entry, not knowing there was an RFC. We did find it (RFC 1078), and Neale and I went back and forth tweaking the code. Eventually I stopped with this:

#!/usr/bin/perl

while(<>) {
  if($_ eq "HELP\r\n") {
    print "tcpmux\r\n";
    exit 0;
  } elsif(lc($_) eq "tcpmux\r\n") {
    print "+OK FINE\r\n";
  } else {
    print "-BLOW ME\r\n";
    exit 0;
  }
}

My friends, that is a fully functional, RFC 1078-compliant, completely secure TCPMUX server, in 11 lines of Perl. Neale has a bash version that he prefers, but I argue mine is better because it's strictly RFC-compliant (only accepts CRLF, etc). To use it, add this to /etc/inetd.conf:

tcpmux stream tcp nowait nobody /path/to/in.tcpmuxd

To use, telnet to port 1. (You can use nc, but you will have to do something like "echo -ne 'tcpmux\r\n' | nc localhost 1" because it will only recognize CRLF-terminated lines per the RFC.) in.tcpmuxd will accept and forward exactly one service, tcpmux. All others will be rejected with a kind explanation. "HELP" will also conveniently list all services it will forward.

You can also test this by telnetting to colobox.com port 1, which is running a fully functional TCPMUX server.

This service has been painstakingly checked for security flaws. A highly skilled team has gone through the entire codebase, line by line, and has determined that there are no known implementation or security flaws in the service. You're welcome.

• 3 Comments
• Posted in Uncategorized

Article posted on Jan 16

oui, an IEEE OUI database search utility

• Posted by Ryan Finnie on January 16, 2010, 4:17 pm

On Friday, I was grepping through DHCP logs, looking for a certain machine, and got sick of going to the IEEE web site to plug in the OUI to figure out the manufacturer of MAC addresses. This involved manually converting a "standard" format MAC address (00:04:f2:e6:93:16, for example) into an OUI format that would be accepted by the IEEE site (00-04-F2).

I found that my workstation already had several OUI databases installed locally (most notably one provided by the nmap package), and hacked together a 5 line Perl script to take a MAC address and use it to search one of the OUI databases. I later fleshed it out into a complete, releasable product. You can download the program from http://www.finnie.org/software/oui/.

At its simplest, give it a MAC address, and it will return a vendor.

$ oui 00:22:19:df:a8:2b
002219 Dell

You can give it multiple items to look up. These can be either a full MAC address or just an OUI, uppercase or lowercase, and can be in a variety of popular formats.

$ oui 00:30:48:88:1B:AF 0004f2e69316 000a.4137.c40a 00:26:99:8d:38:ea 00-50-8D
0004F2 Polycom
000A41 Cisco Systems
003048 Supermicro Computer
00508D Abit Computer

Note that oui was given 5 items, but only returned 4 results. oui searches for several common OUI databases that may be installed locally on your system (the most common would be nmap's database), and they can be quite out of date. Let's rectify that by downloading a current database from the IEEE.

$ oui 00:26:99:8d:38:ea
$ wget -O /tmp/oui-20100116.txt http://standards.ieee.org/regauth/oui/oui.txt
--2010-01-16 15:04:12--  http://standards.ieee.org/regauth/oui/oui.txt
Resolving standards.ieee.org... 140.98.193.16
Connecting to standards.ieee.org|140.98.193.16|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2118408 (2.0M) [text/plain]
Saving to: `/tmp/oui-20100116.txt'

100%[=====================================>] 2,118,408   1019K/s   in 2.0s    

2010-01-16 15:04:17 (1019 KB/s) - `/tmp/oui-20100116.txt' saved [2118408/2118408]

$ oui -d /tmp/oui-20100116.txt 00:26:99:8d:38:ea
002699 Cisco Systems

Much better. If you want to permanently store that database, put it in /usr/share/oui/oui.txt; oui will look there first for a database.

You can also use oui to search one or more organization names.

$ oui -s avaya "university of california"
00040D Avaya
00126D University of California, Berkeley
001B4F Avaya
00E007 Avaya ECS

Let's take a look at how many registrations some companies have.

$ oui -s dell | wc -l
29

That's about 500 million possible MAC addresses, which sounds right for the world's largest PC manufacturer. Let's try my favorite server manufacturer, Supermicro.

oui -s supermicro "super micro" | wc -l
2

Ahh, not so much. What about Cisco? They seem to have a lot of devices out there on the ol' Internets.

$ oui -s cisco | wc -l
448

Wow. That's 7.5 billion possible MAC addresses.

We can also see how many registrations are currently marked "private". These are registrations where the IEEE keeps the manufacturer's identity private for a time, in exchange for a yearly fee.

$ oui -s -c '^PRIVATE$' | wc -l
43

A few notes: First, you will get no results if you just have the nmap database installed, as it uses a condensed format and filters out private registrations. Second, you can use regular expressions to match an organization (PCRE). Third, the -c flag forces the search to be case sensitive.

• Leave Comment
• Posted in Uncategorized

Article posted on Jan 7

Hampr Mobile compatible with iPhone, Android, Pre

• Posted by Ryan Finnie on January 7, 2010, 12:42 am

Two years ago, an "iPhone" version of the Hampr web interface was released, optimized for the iPhone web browser. Well, here we are in The Future, and the same interface works equally well on many WebKit-based mobile browsers. The Hampr Mobile interface has been tested on:

• iPhone (all versions)
• Android 1.6 & 2.0 (including T-Mobile G1, Droid, and the upcoming Google Nexus One)
• Palm Pre/Pixi (and presumably future WebOS-based mobile phones)

In light of this, the interface is now available a https://www.hampr.com/mobile/webkit (the old URL, https://www.hampr.com/iphone, will continue to work). When you log into https://www.hampr.com/, the mobile link at the bottom-right corner of the page will be customized according to what mobile device it detects ("iPhone Version", "Android Version", etc), but the interface itself is the same for all WebKit-based mobile devices.

All phones listed above allow you to bookmark the Hampr Mobile interface and save the bookmark on your home screen. No "App" needed! And of course, while on the desktop, be sure to download the Hampr Firefox extension.

• Leave Comment
• Posted in Hampr

Article posted on Jan 1

Hampr Firefox extension 2.2 released

• Posted by Ryan Finnie on January 1, 2010, 2:39 pm

Hampr is a free, centralized, personal bookmark manager. Hampr is designed for users who want access to their bookmarks from several locations, such as home, work and school. Hampr is similar in concept to del.icio.us, but is not public in nature.

Changes in version 2.2 of the Hampr Firefox extension include:

• Extension is now aware of Firefox's Offline mode, and will not try network operations when Offline.
• A copy of a recent server refresh will be stored locally periodically, and will be used when a new window is opened and the extension is accessed, before the extension has had a chance to get updates from the server.
• Extension has been tested against Firefox 3.6 beta.

Please visit the extension home page and give it a try! Hampr is fully OpenID-enabled; for information about creating an account, please visit the login page.

• Leave Comment
• Posted in Hampr

Article posted on Dec 23

I forgot about Christmas.

• Posted by Ryan Finnie on December 23, 2009, 6:14 pm

Literally.

Ever since a week before Thanksgiving, I've been working on a massive project for work. I'd like to talk about it in detail sometime later because even though it was an exhausting project, it was still very fun and interesting, but that's another post. It all culminated with an 8 day trip to Boston, which I got back from Monday night.

Friday night I was at my hotel, watching TV, when I thought to myself, "Man, they're advertising Christmas shopping earlier and earlier this year." Then I looked at a calendar: it was one week from Christmas.

I was aware of the dates as they passed, but frankly my mind wasn't processing them as they related to holidays, just various deadlines for work. I got home Monday, was still exhausted Tuesday (and the roads were very slick), and now it's 2 days from Christmas.

So yeah, I didn't get anybody anything. Sorry. To make penance, I whipped out the plastic and decided to donate to charity. I had planned on spending about $250, and was deciding between the American Civil Liberties Union and the Electronic Frontier Foundation, but at the last moment I said "what the hell" and donated $250 to each. They both do good work, and I am glad they are there when we need them.

• Leave Comment
• Posted in Uncategorized