Article posted on Aug 29

Introducing 2ping

• Posted by Ryan Finnie on August 29, 2010, 6:28 pm

2ping is a bi-directional ping utility. It uses 3-way pings (akin to TCP SYN, SYN/ACK, ACK) and after-the-fact state comparison between a 2ping listener and a 2ping client to determine which direction packet loss occurs.

No wait, that sounds too dry. I've been writing dry documentation and protocol references for weeks.

2ping is like ping, but does magic.

Has this ever happened to you?

PING 10.0.0.1 56(84) bytes of data.
64 bytes from 10.0.0.1: icmp_seq=1 ttl=255 time=0.062 ms
64 bytes from 10.0.0.1: icmp_seq=4 ttl=255 time=0.064 ms

--- 10.0.0.1 ping statistics ---
4 packets transmitted, 2 received, 50% packet loss, time 3999ms
rtt min/avg/max = 0.062/0.063/0.064 ms

Great, two replies were never received. But why? Did the far end receive the requests and reply? There's no way to know. Enter 2ping.

2PING 10.0.0.1: 64 to 512 bytes of data.
64 bytes from 10.0.0.1: ping_seq=1 time=0.083 ms
Lost inbound packet from 10.0.0.1: ping_seq=2
Lost outbound packet to 10.0.0.1: ping_seq=3
64 bytes from 10.0.0.1: ping_seq=4 time=0.041 ms

--- 10.0.0.1 2ping statistics ---
4 pings transmitted, 2 received, 50% ping loss, time 4005ms
1 outbound ping losses (25%), 1 inbound (25%), 0 undetermined (0%)
rtt min/avg/max = 0.041/0.062/0.083 ms
6 raw packets transmitted, 2 received

Now THAT tells you a lot more. One ping was received by the server and replied to (but you never received the reply), and another was never received by the server. Assuming you trust the other side to have a stable network connection, that suggests both flaky inbound and outbound problems toward your side.

How does it work?

Normal ping uses ICMP. It is assumed that nearly all IP-capable machines are listening for ICMP ping requests, and will respond with an ICMP response.

2ping is a combined client and listener utility that communicates over UDP. You start a listener on the far end (2ping --listen), then on the near end you start the client mode and tell it to go to the far end. In normal conditions, it operates much like a regular ICMP ping.

Client: "Ping!"
Listener: "Pong!"

Actually, it's a little more involved. By default, 2ping uses 3-way pings:

Client: "Ping!"
Listener: "Pong! And Ping!"
Client: "Pong! And that last ping took 1.2ms to complete!

This allows the listener to gain a little more insight into what's happening on the client end. It knows what the RTT is between the 2nd and 3rd legs, and in addition, the client let the listener know the result of the ping between the 1st and 2nd legs. Here's what 2ping looks like in listener mode:

2PING listener (0.0.0.0): 64 to 512 bytes of data.
64 bytes from 10.0.0.2: ping_seq=1 time=0.226 ms peertime=0.077 ms
64 bytes from 10.0.0.2: ping_seq=2 time=0.166 ms peertime=0.035 ms
64 bytes from 10.0.0.2: ping_seq=3 time=0.164 ms peertime=0.032 ms
64 bytes from 10.0.0.2: ping_seq=4 time=1.073 ms peertime=0.040 ms

Nifty, huh? Now the real magic happens when packet loss starts occurring. Say a few replies from the listener to the client never arrive at the client. Communication isn't totally dead between the two hosts, and they do eventually start talking to each other. When this happens, they start comparing notes:

Client: "Ping #1!"
Listener: "Pong #1!"
Client: "Ping #2!"
Client: "Ping #3!"
Listener: "Pong #3!"
Client: "Ping #4! Oh, and I never received a response to ping #2."
Listener: "Pong #4! I received ping #2 and replied. Must be inbound packet loss on your side."

(This is a simplified example that doesn't show the extra communication with the 3-way pings. But the principle is the same.)

Again, this assumes a stable listener's network. "Must be inbound packet loss on your side" doesn't mean much if you can't assume that it wasn't lost outbound from the listener's perspective.

Current status

Go and download version 0.0.1. This version is mostly feature-complete, and implements many of the relevant command-line options from the regular ping utility ("2ping -a" for audible ping, etc). A 2ping manpage is partially complete, and documents all of the command-line options.

2ping is written in Perl, and all of the base module requirements should be in any default Perl install. It is IPv6 compatible with the "-6" switch, but requires IO::Socket::INET6, which is probably not included by default. But 2ping only tries to load it if you are using the "-6" switch, so the module is not needed if you do not need IPv6 support.

This is a VERY early release. The code is horrible and not documented. The command-line switches may change in the future, the wire protocol may change in the future, and the UDP port used by default (currently 58277) definitely WILL change in the future. (I've sent the draft 2ping protocol to the IANA for review, and hopefully they will assign a registered port for 2ping use.) A version 1.0 release will not be made until the code is cleaned up, the final UDP port is assigned, and the protocol is finalized.

• Leave Comment
• Posted in Uncategorized

Article posted on Aug 28

Model M keyboard restoration project

• Posted by Ryan Finnie on August 28, 2010, 5:52 pm

On Monday, I stopped at Twin City Surplus to look around, simply because I was in the neighborhood. I'm not going to Burning Man this year, so nothing really stood out. I went out into the back yard, which contains scrap metal, ammo cans, netting, etc. In a corner of the yard is covered but exposed set of shelves with some old computer and telecom equipment. Now, I'm a computer and telecom geek, but in all the times I've been to Twin City Surplus, there's never been anything worth buying, and most of it just sits there year after year. I'm talking half-broken 1950s key telephone systems, CRT monitors, etc.

But this time, I noticed something. A pile of keyboards, maybe 15 or so. Most were junk, but out of them, I found 3 IBM Model M keyboards, completely caked in dust. I walked out with all 3 for $9 (and a pair of completely black and itchy hands). I then set about restoring them back to their original glory.

As of today, all three are in "usable" states, but I still have a lot of work to do. I've got an order in with Clicky Keyboards for two replacement keycaps and a nut driver required to open the chassis of them to clean better.

Below is a repost of the photos I've posted to Flickr this week. The full set is available here.

Find of the year

3 Model M 1391401 keyboards, found in the outside yard at Twin Cities Surplus in Reno, NV. I picked them up for $3 each. 2 are white logo (1988 and 1989), one is a 1993 blue logo. All have detachable PS2 cables, one is a 10-foot, two are 5-foots. I was able to find all but two of the keycaps. Conditions look good, but they're covered in a ton of dust, so we'll see once I get them cleaned up.

Score!
Read the rest of this entry »

• Leave Comment
• Posted in Uncategorized

Article posted on Aug 23

DNS TTLs and web spiders

• Posted by Ryan Finnie on August 23, 2010, 12:39 am

I've moved a few servers at work to our new datacenter, using the "normal" method: adjust TTLs from 3600 to 120, wait an hour, sync servers, change DNS to new IP, all traffic should be going to new server within 2 minutes, bump TTLs back up to 3600.

Google and Yahoo seem to be compliant, but there are invariably the same 3 bots that appear on the old server up to a day after the switch:

• Baiduspider+(+http://www.baidu.com/search/spider.htm)
• msnbot/2.0b (+http://search.msn.com/msnbot.htm)
• Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)

Maybe there should be an option in robots.txt.

User-agent: *
Honor-DNS-TTLs: no-really-you-should-be-doing-this

• 1 Comment
• Posted in Uncategorized

Article posted on Aug 9

Flickr mail bounce

• Posted by Ryan Finnie on August 9, 2010, 9:37 pm

One of those rare glimpses into the workings of a large public service. Today I sent a photo from my phone to Flickr using their email gateway, and the first time it bounced back:

Received: by www118.flickr.mud.yahoo.com (Postfix)
	id D7723381B9; Mon,  9 Aug 2010 20:40:33 +0000 (UTC)
Date: Mon,  9 Aug 2010 20:40:33 +0000 (UTC)
From: MAILER-DAEMON@www118.flickr.mud.yahoo.com (Mail Delivery System)
Subject: Undelivered Mail Returned to Sender
To: me@example.com
MIME-Version: 1.0
Content-Type: multipart/report; report-type=delivery-status;
	boundary="93240381BA.1281386433/www118.flickr.mud.yahoo.com"
Message-Id: <20100809204033.D7723381B9@www118.flickr.mud.yahoo.com>

This is a MIME-encapsulated message.

--93240381BA.1281386433/www118.flickr.mud.yahoo.com
Content-Description: Notification
Content-Type: text/plain

This is the Postfix program at host www118.flickr.mud.yahoo.com.

I'm sorry to have to inform you that your message could not
be delivered to one or more recipients. It's attached below.

For further assistance, please send mail to <postmaster>

If you do so, please include this problem report. You can
delete your own text from the attached returned message.

			The Postfix program

<photos@www118.flickr.mud.yahoo.com> (expanded from
    <myflickrpostid@photos.flickr.com>): Command died with status 111:
    "/usr/bin/php -d log_errors=On
    /var/www/html/www.flickr.com/mail_handlers/handler.php 2>&1
    |/usr/local/bin/perl /usr/local/bin/log2spread -h -g error". Command
    output: Could Not Connect

--93240381BA.1281386433/www118.flickr.mud.yahoo.com
Content-Description: Delivery report
Content-Type: message/delivery-status

Reporting-MTA: dns; www118.flickr.mud.yahoo.com
X-Postfix-Queue-ID: 93240381BA
X-Postfix-Sender: rfc822; me@example.com
Arrival-Date: Mon,  9 Aug 2010 20:36:31 +0000 (UTC)

Final-Recipient: rfc822; photos@www118.flickr.mud.yahoo.com
Original-Recipient: rfc822; myflickrpostid@photos.flickr.com
Action: failed
Status: 5.0.0
Diagnostic-Code: X-Postfix; Command died with status 111: "/usr/bin/php -d
    log_errors=On /var/www/html/www.flickr.com/mail_handlers/handler.php 2>&1
    |/usr/local/bin/perl /usr/local/bin/log2spread -h -g error". Command
    output: Could Not Connect

--93240381BA.1281386433/www118.flickr.mud.yahoo.com
Content-Description: Undelivered Message
Content-Type: message/rfc822

[Original (truncated) message here]

--93240381BA.1281386433/www118.flickr.mud.yahoo.com--

• Leave Comment
• Posted in Uncategorized

Article posted on Aug 1

Still missing: wife, 2.3 children, dog

• Posted by Ryan Finnie on August 1, 2010, 3:09 pm

As Jack Kerouac said, "If you own a rug, you own too much."^0x00 I think this is one step below that.

I don't need a dining room table. In fact, it's now a hindrance since the dining room area used to be storage for various things (my bike, rock band toys, etc) which will need to be relocated. But I've always wanted a dining room table. Something as a "nice" place, no TV directly in front of you (living room), 80s decor (kitchen), wires everywhere (don't get me started on my home office), etc.

This weekend I went down to San Jose to replace my 5-year-old personal colo server in Market Post Tower. On the way back, I stopped at Fry's in Fremont and browsed for awhile, eventually walking out with a portable keyboard and some soda for the trip home. I didn't plan on visiting Ikea, but between Fremont and Sacramento (I took 680, so I didn't pass by the Emeryville Ikea), I remembered that I had recently broken a bowl from my Ikea dish set, and replacing it was a flimsy excuse to stop.

I walked in for a $1.50 bowl and left with a $350 dining room set.

The table is nice. See, Ikea has two types of items: cheap and will fall apart in an hour, and reasonably priced and decent but not top notch quality.^0x01 I'm hoping this set falls into the latter category. It's obviously not real solid hardwood, but the laminated covers seem to be decent quality. It can be configured as a round table (not pictured), or you can release some tension latches under the table, pull the two halves apart, and unfold a self-contained leaf for expanding it to an oval shape (pictured). The tension latches are then re-engaged, which prevents the ends from inching away from the leaf.

By the way, you'll notice only two chairs. I wasn't sure if I could fit the table plus all four chairs into the Prius hatchback area, so I only picked up two. Turns out I could have, but I'll just pick up two more the next time I'm in Sacramento. And of course I don't plan on leaving a full service setting out all the time.

You know, now that I think about it, a rug would look nice in there.

^0x00 I'm not entirely convinced he actually said that. I can't find a direct attribution, just people who start out, "As Jack Kerouac said..."

^0x01 Rambling sidenote, which references spoilers in the Millennium trilogy. If you haven't read it (I would highly recommend you do) but would like to, stop reading now. It's interesting knowing that many Swedish people furnish their homes entirely with Ikea. In the beginning of The Girl who Played with Fire, Lisbeth Salander, who has stolen billions from a now-dead criminal, has covered her tracks but is lying low. Under a fake identity, she buys an enormous penthouse suite, but then goes to Ikea to furnish it. In my world, a "fully Ikea-furnished apartment" is the realm of the poor college student living in a 200 square foot apartment. In Sweden, apparently even billionaires shop there. Granted, there are some part of Ikea I drool over. If I ever have a dream home, I'll probably build most of the kitchen from Ikea fixtures. But there are some areas I just don't like. I know many rave over their beds, but I have two problems with them. First, I have yet to find an Ikea bed that I didn't think was ugly (you're not likely to find something like this at Ikea). Second, I've never been a fan of the European style mattress (slatted frame, no boxspring). In summary, Ikea is good for some things, but I can't see furnishing an entire home with it. Okay, rant over.

• 1 Comment
• Posted in Uncategorized

Article posted on Jul 23

Ads begone

• Posted by Ryan Finnie on July 23, 2010, 10:43 pm

I have removed all Google ads from finnie.org, after logging into my AdSense account and noticing the money it brought in last month.

$0.02. Two cents. Interestingly, on zero clicks. (Pity payment?)

Meanwhile, velociraptors.info brought in $79.15, and is now result #6 on Google for "velociraptor". That's above even any of the Xkcd comics which velociraptors.info parodies. Yeah, not removing ads on that.

• Leave Comment
• Posted in Uncategorized

Article posted on Jul 23

Ninja-like speed and stealth. Free soda refills.

• Posted by Ryan Finnie on July 23, 2010, 6:24 pm

Panda Express is a well-oiled machine, designed to make sure you always get what you didn't want.

Once every week or two, I pick up dinner at Panda Express on the way home. I usually get the rice bowl with mandarin chicken, and an order of rangoon. The rangoon is served with a small side of sweet and sour sauce poured into a plastic cup.

I've never been a fan of sweet and sour sauce, so I used to say I don't want the sauce. But it usually ended in failure. The problem is someone behind the counter would walk by my order as I was waiting for the cashier, see an order of rangoon without the cup of sauce, and think, "oh, someone forgot to pour a cup". Sometimes I'd be able to catch that person in time, sometimes not. Sometimes it would happen multiple times by the time I get to the cashier. Usually I left with a cup of sweet and sour sauce. Eventually, I gave up on the matter of asking altogether.

Today, the person making my order forgot to pour a cup of sauce. Success! It was a slow night, and there was nobody in line ahead of me, so it was quick to the cashier. I paid and left the restaurant, basking in my victory. Granted this was because of inattention by the woman making my order, so it was a hollow victory, but I'll take them any way I can get them.

I got home and opened my bag. Staring at me was a cup of sweet and sour sauce. She must have poured it while I was reaching for my wallet. Crafty ninja.

• Leave Comment
• Posted in Uncategorized

Article posted on Jul 17

404, revisited

• Posted by Ryan Finnie on July 17, 2010, 4:50 pm

In 2002, I wrote a 404 parody of Poe's The Raven. Overall I liked it (particularly the rhyming of three 4-syllable words), but it never felt completely right, particularly the 2nd and 5th lines. Today, I found my original post, went back to the drawing board -- so to speak -- and ended up with this.

Let us examine the original. Most people remember The Raven as 8-8-8-7 verse with a "da-da" rhythm. But look at what the first two lines actually are, with what I believe are the intended stresses:

Once upon a midnight dreary, while I pondered weak and weary,
Over many a quaint and curious volume of forgotten lore,

It's actually 8-8-17. Now, you could ignore "over", at which point it would be 8-8-8-7, but would put a lot of stress on the middle syllable of "curious", which doesn't sound right. What I believe Poe did was to break up the mental rhythm the reader had gotten himself into over the first line, and restore it by the end of the second line, and the entire line was meant to be read more or less free-form. The lack of a "-ry" rhyme at the word "curious" 8 syllables in seems to support that.

Now here's my original version, with stresses on how it ended up being read:

Once upon a midnight dreary, while I searched on Google, weary,
sorting through a list of results; Middleeast histry and Al Gore.

The problem was I continued the "da-da" rhythm and ended the second sentence's first fragment after 8 syllables. There may be 17 syllables total in my second line, but because the rhythm wasn't broken up, the reader continued with the rhythm and ran into a big problem with the second fragment. He would then try to compensate by cramming "Middle East" and "history" each into one syllable to preserve the 8-8-8-7 rhythm.

This is the replacement:

Once upon a midnight dreary, while I searched on Google, weary,
scanning through pages of random results from São Paulo to Al Gore.

This works better. There are still 17 syllables, but I used some of Poe's same tricks to break up the rhythm into a free-form line. The stress positions aren't exactly the same as The Raven, but it didn't have to be, as long as the rhythm was broken up by the 8th syllable, and the sentence ended on a stress.

Other changes to my parody were comparatively minor. The third line had a lead-in "and" while the original did not, so I changed "And there it was" to "Suddenly". The fifth line actually had an error, 16 syllables instead of 15 in the original. So it was rewritten to fix this, and it now contains 15 syllables, but also matches the rhythm of the original closer. The number of syllables in each fragment of the sentence match exactly, though the stresses aren't the same between the two because I couldn't find a satisfactory way to end on a one-syllable "-ore". But again, it breaks up the rhythm and restores it again by ending on a stress, so it works.

(Note: I am not a literary expert, but I play one on TV! Edgar Allan Poe could not be reached for comment.)

• Leave Comment
• Posted in Uncategorized

Article posted on Jul 12

Kindle reading list

• Posted by Ryan Finnie on July 12, 2010, 12:07 am

In early January, I bought a Kindle 2 "International Edition" (previous editions used Sprint EVDO for wireless transfer; Amazon moved to GSM late last year). I had been considering an e-reader before then, and was mostly leaning toward a B&N Nook.

As a whole, the Nook and the Kindle are fairly similar. Both are backed by large libraries, have wireless or USB upload capability, use the same e-ink technology at the same size (ignoring the Kindle DX's larger screen), and cost the same.

The Nook has a small color touchscreen at the bottom for navigation, includes wifi connectivity, and includes ePub support. The Kindle 2 is usually slightly faster, is noticeably thinner and lighter, includes a rudimentary 3G web browser with free access, and includes a physical keyboard area that doubles as a palm/thumb rest (the keys are resistive enough that you can hold the device with your palms and thumbs gripping the keyboard area without fear of pressing buttons).

Ultimately, I chose the Kindle for two reasons. First, the Nook was backordered and wasn't expected to ship for well over a month. Second, I had just received a $250 Amazon gift certificate, which neatly covered most of the $259 price tag. (The price is now $189. C'est la vie.) Wifi would have been nice, but is largely unnecessary for book transfer between 3G and USB. Also, on principle I'd like to have ePub format support, but it's also largely unnecessary -- my two sources for books are the Amazon store and Project Gutenberg, and Project Gutenberg now has Mobi (the Kindle's native format) support.

I was a bit hesitant about buying an e-reader. I've never been much of a reader. Sure, I read a lot of research, papers, etc as part of my job and computer lifestyle, but I would rarely read novels. It's not that they bored me or anything of that nature, I just wasn't into curling up with a good book.

Looking back, it was the book part of the equation that was the problem, not the reading. I've never liked the ergonomics of the physical book. The Kindle solved that problem.

I immediately took to the Kindle. It feels very natural in the hands, the perfect weight and size. The battery lasts practically forever -- I only turn 3G on when I need to download a queued book, and with 3G off, it can easily last a month on a single charge. The screen is the perfect size and the perfect look. When I first saw a 16-shade e-reader in person (a Nook at a B&N display counter), I thought it was a non-functional display model; it honestly looked like the words were printed directly on the display area. There is a refresh between pages where the screen goes entirely black and then clears before the words are written, and while it initially was quite noticeable, it's now natural -- as if a subliminal cue to begin again at the top, the same as when you're turning a real page -- and I don't notice it anymore.

I've been reading an hour or two many nights before bed; sometimes even longer, to my detriment. (The Girl with the Dragon Tattoo is somewhat long and pretty slow in the first half, but picks up quickly. I spent about a week on the first half, and read the second half in one sitting, until 6AM.)

Of course no "e-reader" post can be complete without mentioning the iPad. I played with one when it first came out, and even if I didn't already have a Kindle at the time, I wouldn't have considered it as an e-reader. It's too big, too heavy, and its LCD is not suited for reading books, especially compared to e-ink. Its one redeeming factor is, because of the full-screen touchscreen, page seeking is much more intuitive. Because the Kindle is entirely e-ink, you lose quick random access ability. Say you want to find an earlier passage, and you know it's roughly 25% into the book. A physical book is the fastest: you just turn to a spot roughly a quarter way in and quickly go back/forwards from there. The iPad is a close second: drag your finger to the approximate area. The Kindle is the slowest: you either have to rely on previously bookmarked pages or passage highlights, or enter a numeric position identifier (say 1250 out of 5000), then go backwards or forwards from there. Pages take about half a second to turn on the Kindle, which is fine for sequential reading, but worse for random access seeking. Half a second is about how long it takes to flip a physical page, but the distinction is you can flip 2 or 10 pages at a time with a physical book, but not with a Kindle. Still, the e-ink technology is such a vast improvement over LCD for reading, I would not consider an iPad for an e-reader.

It is now 6 months later, and I'm currently reading book #20. It's mostly been a mix of crime mysteries and science fiction books, but I'm keeping my options open. I have yet to read a book that I genuinely didn't like, though two stand out for me. The Brick Moon is an 1865 short story that, while an enjoyable story, was written in a dialect that was quite difficult to read by today's English. And while Mostly Harmless -- the final Douglas Adams book in the HHGG series -- wasn't bad, it was incredibly depressing, and was a stark contrast to the irreverence of most of the rest of the series.

Below is what I've read so far, in reading order. I'll probably continue updating this post as I finish new books.

1. Freakonomics, By Steven Levitt / Stephen J. Dubner (revised edition)
2. The Nine Billion Names of God, by Arthur C. Clarke (short story)
3. The Hitchhiker's Guide to the Galaxy, by Douglas Adams
4. Star Trek Generations, by J.M. Dillard (novelization)
5. Starship Troopers, by Robert A. Heinlein
6. The Restaurant at the End of the Universe, by Douglas Adams
7. Red Dragon, by Thomas Harris
8. Snow Crash, by Neal Stephenson
9. Life, the Universe and Everything, by Douglas Adams
10. The Wonderful Wizard of Oz, by L. Frank Baum
11. So Long, and Thanks for All the Fish, by Douglas Adams
12. Mostly Harmless, by Douglas Adams
13. Cat's Cradle, by Kurt Vonnegut
14. The War of the Worlds, by H. G. Wells
15. The Girl with the Dragon Tattoo, by Stieg Larsson
16. The Brick Moon, by Edward Everett Hale (short story)
17. Hide, by Lisa Gardner
18. Nineteen Eighty-Four, by George Orwell
19. The Girl Who Played with Fire, by Stieg Larsson
20. From the Earth to the Moon, by Jules Verne
21. Around the Moon, by Jules Verne (direct sequel to From the Earth to the Moon)
22. The Girl Who Kicked the Hornets' Nest, by Stieg Larsson
23. Anthem, by Ayn Rand
24. Alice's Adventures in Wonderland, by Lewis Carroll

• Leave Comment
• Posted in Uncategorized

Article posted on Jun 7

HOWTO: Delete a 3ware DCB

• Posted by Ryan Finnie on June 7, 2010, 1:51 pm

I have a large enough infrastructure at work where we're often (once or twice per month) moving disks around, replacing failed disks on 3ware controllers, etc. One thing that often shows up is issues when an unwiped disk previously used in a 3ware controller is put into another 3ware controller. First, let's step back:

Update (2010-06-08): I added/updated some information based on what I've found analyzing a handful of DCBs in the last few days.

3ware DCB analysis

LSI 3ware (née AMCC 3ware, née 3ware) uses a proprietary disk layout that stores RAID information on the disks themselves. This has the advantage of being able to replace/upgrade a controller without having to reconfigure it. However, if you move a disk that was previously part of another array into a new machine without wiping it, it shows up to the new array as a failed existing array.

3ware stores this information on a disk in a section called the Disk Control Block (DCB). There are two versions of the DCB which I refer to as Old style and New style. On both Old and New, the DCB is stored at the end of the disk. I have discovered that the DCB appears to be 1024 LBAs long (1024 * 512 bytes per sector = 512KiB). Additionally, the Old style includes a copy of the DCB as the first 1024 LBAs of the disk. The actual disk data starts at LBA 0 on New style and LBA 1024 on Old style.

The Old and New style DCBs' 512KiB blocks are laid out differently, and I think I have found magic strings that can differentiate between them. On Old style, there is the string \x03\x00\xe9\x07 2044 bytes into the DCB, while New style has the string '3WareDCB' 2056 bytes into the DCB.

All 9000 series controllers I've seen so far seem to use New style, and until recently, all 8000 series controllers I had seen had Old style. I thought it was an 8000-versus-9000 thing, but I recently came across an 8006-2LP 2-disk RAID1 array that had a New style DCB. Updated firmware perhaps? Even odder about this array was that on all the New style DCBs I had seen until then, the actual disk data began at LBA 0, that is the beginning of the disk. However, on these disks, the disk data began at LBA 8. LBA 0-7 (4 KiB) contained something that sort of looked like a DCB (serial numbers are in there), but is much too short to be a proper DCB. This suggests that, at least with New style DCBs, the beginning data offset is stored somewhere within the DCB, but I have not yet been able to find it.

It should be noted that the DCBs on 8000 and 9000 series controllers are not compatible with each other. Plugging a disk previously used in an 8000 series controller (and not wiped) into a 9000 series controller will produce the dreaded UNCONV-DCB error. The disk is still usable as a (new) member of an existing array on the 9000, but it appears you cannot get rid of the UNCONV-DCB error until you take the disk out and manually wipe it. 9000 series are backwards and forwards compatible with each other, so you could take the disks from, say, a 9650SE and put them in a 9550SX or vice versa.

The previous paragraph's information comes from 3ware's docs and my personal experience, but that recent discovery of an 8006-2LP with a New style DCB throws a wrench into things. It could be that 3ware is actually referring to Old and New style DCBs when they say 8000 and 9000. If that's true, I may be able to put those New style 8006-2LP disks into a 9000 series controller. I'll have to try that sometime.

Wiping the DCB

So, the important part to take away is if you want to reuse a disk that was previously in a 3ware controller on another 3ware controller as if it were a blank disk, it is best to wipe the disk's DCB areas first, to make sure the new controller doesn't recognize it. I'm currently writing a utility that can search a disk for DCBs, tell the difference between Old and New style DCBs, dump them to files, wipe them, etc. But for now, here's a short Linux shell script for wiping the DCB areas from a disk:

DISK=sdz
LBAS=$(cat /sys/block/$DISK/size)
dd if=/dev/zero of=/dev/$DISK bs=512 count=1024
dd if=/dev/zero of=/dev/$DISK bs=512 seek=$(($LBAS-1024)) count=1024

Replace "sdz" with the device name (triple check; I'm not responsible for you hosing your boot drive). You can skip the first "dd" command if you are sure that the disk came from a disk with a New style DCB, but if you're "wiping" the disk with the intent of reformatting, it doesn't hurt to erase both the beginning and ending 1024 LBAs.

• 1 Comment
• Posted in Uncategorized