Posts by Ian

ZFS has value outside arrays

My laptop runs Open Solaris, and ZFS provides some real value. It makes upgrades incredibly safe: you take a clone, upgrade the clone (which only requires the space of the changed files), boot off it, run with it for a while and finally promote the new version to replace the old. By making filesystems cheap, you can put each lump of the system into a separate filesystem to include it in, or exclude it from, the upgrade process (I can boot my laptop on 2008.11 or 2009.06 and see the same home directory: yes, I know this is possible on other filesystems, but requires hard partitioning).

Plus checksums to guard against failing drives, compression selectable on a per-filesystem basis, snapshots for backup...

It's all old news

I went to the meeting last year, ranted at the speakers, and then went back to Simon Davies' flat and ranted at Mark Burgess (nice bloke) and his sister who is their marketing chief (nice woman).

But then I got a clue and just switched ISP to one that's not involved in the trials. Doing that flushed out a few ISP-dependencies in how I was operating, which mean that I would now find it much easier to switch again if I needed to. Well done BT: ditching you as an ISP, plus the opportunity to rationalise some other spending I do with you, chopped £350 per year off your revenues: how many Phorm users will it take to replace that?

So I didn't bother going this year. It doesn't affect me immediately, it's likely they'll run out of cash before BT stir up the hornets' nest of deploying and I now have the option of just running an IPsec tunnel to a suitable proxy if I have to. Duke Nuke'em will be released before it's a real risk.

Homophone Corner, Phonetic Spelling

formerly, not formally.

reckless, not wreckless.

Nathan Barley Ahoy!

Youtube that bitch to escape velocity.

Oh, bits of stuff

I've just this very day flown home from meetings in San Jose: it has to be said the traffic is very light. I measured the dot com bust by how long it took to drive from my preferred hotel by Palo Alto Caltrain and Sun up by the Dumbarton Bridge, and down in San Jose it all felt similarly screwed. Stanford Shopping Centre was empty yesterday, but there was a queue out the door at the Redwood City Fresh Choice. Google engineers may find themselves in a buyers' market.

On the Stevie Nicks front, the only `making of' documentary worth watching is the one that FM did for their recent reformation album. There's a Google Lesson there, which is that basically nice people (as they all appear to be) can have second lives.

Long finished issue

I've lived adjacent to them for more than forty years, and Kings Norton and Kings Heath have alway s been spelt without an apostrophe. The thought that Acocks Green (and would that be Acock's or Acocks', eh?) or Kitts Green might need an apostrophe has never even occurred to me, and I do grammatical pedantry with the worst of them. Druids Heath, Highters Heath...

Here's a photograph of Kings Heath station in 1957; fifty years ago the apostrophe was gone.

http://www.photobydjnorton.com/RailwayPictures/KingsHeathBoxClr.jpg

And more fun, are we now to worry about introducing a space into Kingstanding?

Cheap laptops on E Bay Ahoy

It would be interesting to use the FoI Act to see how many of these laptops are ebayed and reported lost.

Why allow password authentication?

``To access my controlled servers requires a nice long password and originating from a 'trusted' network.''

Surely to God people aren't running ssh listeners with password authentication exposed to the Internet? Under what circumstances couldn't you use a keypair and massively reduce the risk? The quality of the passphrase is a slight issue, especially if there are offline attacks, but that all presumes that someone has a copy of your private key. That massively reduces the risk compared to using a password for straight username/password authentication.

Visible on BT Connection

The original album cover is visible in the wikipedia page from my BT connection this morning. Which is odd.

Hitler

``What's next removing Mein Kampf from the English Lit course because they've suddenly realised a Mr A Hitler might not have been a nice chap after all?''

In order to be removed, it'll need to be on a course to start with: could someone tell us where that's happening? I'd be surprised to find that any GCSE, GCE, CSE, CEE, School Certificate, Pre-U or other qualification in English Literature has studied Mein Kampf.

Better than mine...

I spent Friday at a regional development board conference, where I was one of the 20% that wasn't working for the public sector. And in a city where more than 50% of the population are from ethnic minorities, less than 5% of the attendees were.

The most inane comment came from a media bloke who assured us that his son, ten, didn't wear a watch because he'd far rather carry a more capable device, ie a phone, and therefore only old people would wear watches. Presumably that's so he can get twitter, too.

Goodbye BT

They phoned me up last week, ignoring the TPS, to ask me if I wanted to re-sign my contract at a two quid per month discount. I declined, citing the impending Phorm roll-out (had I been smart I'd also have complained that the much bruited Webwise phishing protection hadn't been delivered, but that might have been a bit too subtle). They said `OK' and moved on to the next mark.

Enjoy

It's a pretty good production. I saw it at Stratford --- sorry, no queuing, I just filled in my full member's form and received row C for my daughter and me by return of post --- and rather enjoyed it. It's been cut heavily to fit in three and a half hours, which is good --- compare Dear Ken's production in the early 90s, which was nearly an hour longer.

Tennant is surprisingly good --- remember, Hamlet is a young man's part, and his Doctor has been an audition for Hamlet from the off --- but Stewart steals the show as Claudius. He's old RSC, and can actually speak the verse, and that brings out of Maria Gayle as Ophelia (a name to watch: she did a great Miranda a couple of seasons ago and she's sensational here, and looks great in her bra and knickers too) and especially Penny Downie as Gertrude (again, old RSC: she played Lady Anne opposite Sher's Richard III amongst other things) fine, fine performance.

You won't come away having learnt something about Hamlet, either the man or the text, that you didn't know before, but you'll have laughed (a lot: it's _funny_, with Oliver Ford Davies as Polonius timed to perfection) and seen some great performances. The whole thing is slightly less than the sum of its parts, but it's a fine evening's theatre.

It won't be one of the great RSC things that's remembered for decades to come: my wife saw the Sher R3, we both saw the Stephens Lear and the Brannagh Hamlet and the Mendes/Russell-Beale R3 and the Juliet Stevenson Becket Shorts, I saw all eight of the Histories in four days over Easter.

But if you don't know the play it's a great introduction. I think the young might find it all a bit too angsty. My daughter, 12, isn't a Shakespeare neophyte --- to the amusement of people sat near us she was able to compare Patrick Stewart's Claudius with his Hamlet at Chichester and his Prospero in the old main house --- and she was happy to see Tennant, especially two feet away on the walkway. But she enjoyed the Macbeth more, the motivations being a lot clearer (and, to be honest, the production a lot better: Goold is a real rising star). And if you do know the play, it's light, it's fun, it's well acted, and at worst you'll be left with a slight feeling that there's more to be had from the text.

Headphones; Recommendations

It's hard to claim that recommendation-based systems are new enough to be copied from anything in specific. Wasn't the originator called Firefly, run out of MIT, and rolled up into the Amazon recommendations engine (which is actually rather good)?

On the headphones front, $80 is loose change. I paid about $120 for my Shure e3cs (they're meant as stage monitors, and they're an awkward impedance for little battery powered devices, but they're fine with an iPod so long as you don't want to operate at high volumes). Recently I mislaid them and, thinking I'd lost them, bought some some Shure SE420s which were about £170 (yes, pounds this time). They're fabulous, and because of the isolation they provide you don't need high volumes to overpower background and no-one else can hear your music at all. My 3cs turned up a few months later, so now I have a spare pair...

When does spindle count, er, count?

Honesty in Commenting: I'm a happy Pillar customer, although I'm also very fond of my EMC storage as well, and Mike's one of the most engaging men I've had dinner with in recent year. However, what follows is my long-standing take on storage, formed more during my days as an Auspex customer.

Spindle count matters (ie you need more of them) when you either have a random read rate greater than the aggregate operation rate of your drives, or you have a sustained write rate ditto over a period of time longer than you can solve with array cache.

Back in the day, central storage took a pounding on the read side, because clients were mostly very memory poor. Read was hard to optimise away: if it's not in cache (and it usually wasn't) you have no choice but to go to spindle, and then you need lots of them. So my Auspexes used to be something like ~70% read, and most of those reads were serviced from disk. 84 x 36GB RAID5 seemed like a good way to provide ~2TB (or, for those with very long memories, 60 x 1.3GB RAID 0+1 seemed like a good way to provide ~40GB).

But these days, a not-dissimilar workload is write heavy. The clients have plenty of RAM which means that they rapidly stabilise at a point where the read load they impose is relatively small, and most of the central disk time is spent coping with writes --- the clients, properly, issue writes as soon as they have dirty pages. Write can always be reduced to a cache operation, so long as you have enough cache, and RAM --- even mirrored ECC RAM with battery back-up --- is dirt cheap.

So yes, if you have a burst of write which exceeds the capacity of your cache to the point that not even the portion that will fit into your cache helps, and not even the write re-ordering that the cache allows helps, you'll need more spindles. But my experience of sizing storage is that most people grossly over-estimate the scale of their problem in terms of duration: you may need oodles of random write performance over a few gigs, or perhaps a few tens of gigs, rarely more.

If you need to do random writes over many hundreds of gigs such that you can't re-order them in any useful way, then you're going to need spindles, and lots of them. Even then it's an open question if in that environment you're better off with N 300GB FC disks or 2N 1TB SATA disks short-stroked to 300GB. Mike would argue the latter, and he'd argue that you can probably get some mileage, if only for snapshots and long-term archive, out of the 700GB that are not part of your short-stroking.

So when I look at my arrays (fairly intensive Oracle workloads on some, Clearcase on others) I don't see the limiting factor being the spindles: I see it being the ability of the controllers to keep up with the shorter bursts. And the easiest way to improve that is to throw more RAM at the problem. Spindles may allow more operations to complete within 7ms, but they still won't improve the speed of any individual operation: if it's going to disk, it's going to take 7ms. RAM allows operations to complete (essentially) in zero time. 100 lorries can carry more goods up the M6 than 1 lorry, but it'll take just the same length of time for a given package to get from Rugby to Carlisle.

Now this analysis doesn't help for read. But if you can do your read in a reasonably sensibly ordered way (as, say, during backup or large table reads), then 100 spindles of 1800rpm ESDI is more than enough bandwidth to be going along with, never mind 100 spindles of anything remotely modern. If you're doing random reads, not so much, and then I agree you need the fastest disks you can get.

But those random reads are again going to be taking 7ms each, and I would seriously question the overall design of a production environment in which you need to sustain 10K random reads per second. Yes, 100 FC disks will do it, and 100 SATA disks will struggle, and yes the latency will be lower with FC, but still...7ms, 10ms, both are lifetimes in terms of CPU speed, and isn't it God's way of telling you to either put some indexes on your tables or just buy the terabyte of RAM you know you want for your application?

ian

Handy things to know about

27001 and 25999. And if you've actually done a complete certification cycle in one (or better both), that's good.

IBM && Complexity Theory

``I once received several empty boxes from IBM. Ostensibly they were used to fill the spaces in a larger box in which was a couple of cryptographic cards, but the packing system was obviously clever in knowing how many boxes were needed inside the box.''

Good to hear that a classic NP problem (the knapsack problem) is affecting crypto people.

Biased Language

``Four in five parents who failed to apply internet controls''

Failed to? Or just didn't? Or actively chose not to?

oracle of t2000 is fine for us

We're running all our Oracle on T2000, with (for extra points) everything in Solaris 10 zones and all the storage coming over NAS from one of Uncle Larry's Pillar boxes. All seems very stable, and plenty fast enough for our purposes. The power savings are non-trivial too.

Material Change to Terms and Conditions

``I assume the next step for BT is to make the opt-in tick box a requirement in the contract to access the service (or maybe they will just put you a higher price if you don't opt-in). BT is just beyond shame.''

They could try either of those things. They'd be hard-pushed to argue it's not a material change to the terms and conditions, though, which means that their hard-won long-term contracts for their customers are worthless. It's like cats in front of fires: hold their tail and they want to go somewhere else, irrespective of how happy they are where they are.

I told 'em

I wrote to my MP when the ludicrous proposal was first made, and got a patronising ``experts say it'll work'' response from the hilariously named Paul Goggins. Nice to see I was right and they were wrong.

RMS at Number 10

The sight of Billy-boy, blinking away alongside Tony Blair, each unable to decide who was the more star-struck was moderately amusing. But having spent a week with RMS lo these twenty years ago, and having met ESR a couple of times, I am looking forward to seeing Cameron holding a similar meeting surrounded by Stallman, holding a recorder, and Eric, holding a .44 Magnum...

Grauniad Sex Therapist

But for us men of a certain age, being able to read Pamela Stevenson's advice on anal sex has a certain frisson.

Former Sendo Site

I presume that this is the building on the A45 as you approach the airport. In which case, it's the former Sendo operation, so it's already been through one death and resurrection.

How to fight back

One option is to change ISP. A more effective one is to boycott websites that pay Phorm: their main source of revenue. I'll miss the Guardian's website, but it looks like I'll be paying it my last visit.

BT CEO Mail, @mark

If they really said ``I would like to assure you that the proposed service is an “opt in” service.'' then that's a massive change of position. If CPW and BT go for an opt-in scheme (ie default opt-out) and Virgin join them then the game is essentially over for Phorm.

CPW: The Smart Ones

At the moment there are three classes of ISPs: those that are pimping data, those that appear to be saying things to avoid pimping data, those that are saying nothing. CPW are the only members of the second category: Phorm may claim them as a customer, but if there's no kit installed and they're already talking about an opt-in scheme (presumably Phorm offer a revenue split). So CPW are positioning themselves neatly to capture the diaspora from BT: unlike random ISP X who might sign their souls to Phorm the day after you switch to them, CPW are saying clearly (a) opt-in and (b) those that don't opt-in will be fenced off so their data doesn't hit Phorm's boxes.

Since the set of people who will opt-in is small, CPW have just for practical purposes announced they reckon recruiting Phorm refusnik's from BT is a better business model than taking Phorm's money. I reckon that's the end of Phorm, and BT's next move will be interesting and crucial. That BT have been caught lying about trials in the summer won't go well for them, either.

Stock price down 30%, one major `customer' distancing themselves and another being neutral means that BT are now the only major customer of a shareholder toxic, PR toxic spyware company. Not a nice place to be.

What are all the documents for?

``any company that uses google apps is heading down the bog''

Walking into any office today, you see desks from wall to wall occupied by people typing. Occasionally they hold meetings or use the phone. Most of the time they type.

Thirty years ago, perhaps ten percent of the workforce, if that, would have been an audio- or shorthand-typing pool. The rest would have held meetings, taken decisions, done real work. And gone home at 5pm.

Now they all type. And go home to a laptop and a blackberry, on which they type until they fall asleep.

Do you believe the problem is that the tools they are using to type aren't sharp enough? That they can't type quickly enough (because most of them type very badly, it has to be said)?

Or is the problem that they're typing at all? That what were once hand-written memos, taking a few seconds, are now beautifully produced multi-font structured documents with indexes and branded headings? That people seriously --- indeed, I was flamed on Slashdot for claiming otherwise --- believe that the formatting of internal documents has business value?

We've replaced thinking about business with thinking about word processing. The very idea that the brand of word-processing software affects the value of the work done with it is laughable. The competition isn't about Arial vs Helvetica or Calibri vs Frutiger; the competition is between profitable and not profitable. And pissing around all day with word processing of internal documents is Not Profitable.

A company that uses Google Apps just saved a few hundred pounds per employee, and reduced the turnover of computing equipment. What's the downside?

Disks are disks, in the end

I have a farm of about 200TB of disk in a variety of arrays, from EMC, DotHill, Pillar and others. The drives range from 7200rpm SATA, in the one of the EMCs, the Pillar and some OEM Adaptec storage, through 10K SCSI in the elderly DotHills and on through 10K FC in the remaing EMCs. We don't do 15K because an 8-way assemblage of 7200rpm is enough performance on the read side and our writes are bursty enough that the battery-backed RAM on the arrays isolates us from write performance.

The reliability appears totally unconnected to the brand and specification of the storage on the sample sizes I have. The Pillar has 100-odd 500GB SATA spindles, and (reaches for large pieces of wood) hasn't dropped a spindle in 18 months, for a truly awesome MTBF. It supports a large Clearcase environment and a whole bunch of Oracle databases, so it takes a pounding. The EMCs, on the other hand, have suitably fairy-dust sprinkled FC drives, and at one point we were on first-name terms with their maintenance people. That, and the hideous botch on Clarrions where the failure of one of the leftmost disks causes the write cache to stop working, made for some entertainment for a few months. We'd had a bad batch: of course you tend to get all the disks in an array you buy from a single batch.

The fun with the EMC SATA is that rebuilding an 8+1 RAID5 group to a hotspare with a centralised RAID controller is Not Quick: the Pillar improves on this (we've tested it) by having a RAID controller on each shelf to it can saturate the hot spare, but nonetheless rebuilding 500GB of parity is not rapid.

But for the Time Capsule, the whole thing seems moot. Unless you have 20 small laptops, the duty cycle for a Time Capsule will be a spin up, some writes and a spin down a few times an hour. In fact, a disk designed for 24x7xpounding may not be the best bet: a drive optimised for frequent spin down, like a laptop drive, might be a better fit. Six 120GB SATA 2.5" laptop drives in a RAID6 assemblage would be the thing, wouldn't it?

Hard to see how it's not Interception

It's Interception, per RIPA.

There are immense data protection issues.

Anyone whose computer is taken in a police raid is subject to having the cookie in their browser correlated with the cookie in the Phorm cache.

There's also a copyright issue: if they're taking responses, any website for which I pay is having its content stolen by a third party. And if I'm paying for content, and it's modified in flight, who's liable?

The critical difference is...

...that the Tories had good old British stiff upper lip, not a few of them had been in the military, and dusting yourself down and being stoic was the done thing, even after the Grand Hotel. When Brown and Blair hear reports of things happening, their first reaction is to shit their pants, and once they've cleaned that up they run, screaming, into the nearest available bunker to huddle with the rest of the cowards that pass for our `leaders'. The Labour Party used to have the likes of Denis Healy, who was a beachmaster at Anzio. It now has people whose only contact with uniform is denying them money and then sending them to die overseas.

I was on a plane inbound to LHR when the IRA mortared it in the early 90s. We diverted to STN, got a bus back to the LHR carpark, and got on with our lives. I lived in Birmingham through the 1970s: after the pub bombings we just got on with things. New Labour: New Cowards.

DNA?

On what basis did they take his DNA, one wonders?

The current stuff works well, so it's promising

We've got ~40TB on a NAS Pillar, and we're using it to serve everything from mail via Cyrus (lots of small files, mostly read) through home directories and Clearcase up to Oracle. The QoS stuff they offer does what it says on the tin, clamping the ability of the people that didn't pay for the box to impact on the people that did, and giving everyone predictable performance. There's a few matters of taste (for example, rather like contended DSL when there are few users, giving low priority users the whole system when no-one else is using it doesn't manage their expectations very well, so it would be nice to clamp their performance even when there's plenty spare) but the box basically does what it say on the tin.

Throw in stone-reliable hardware and a software architecture which means that problems --- and you don't buy startup NAS hardware without expecting and tolerating a few of those --- simply trigger a second or so of failover delay and it's a sound product. And well priced, too.

I'm fairly hard-bitten with NAS products, having been using NFS as my dominant storage technology since 1986. I've run Auspexes and NetApps, along with Sun on the front of EMC, Datacore and other hip buzzwords. But the Pillar has been at least as good as the best of the rest, and in terms of having one vendor to deal with for the whole stack harkens back to the heyday of Auspex (not surprising, given the number of Auspex alumni at Pillar, and standing an Axion next to an NS6000 is a hoot).