System is SCO Unix 3.2.4.2.1 - updated once from 3.2.4.2.1 6 or 7 years ago. 24 ports - every one dedicated to a user or printer.

RAID controller failed a year ago, and it screwed up one of the drives beyond ability to rebuild, so it was running on one drive and tape backup. QIC backup system failed a week ago, so we put in a DAT drive and bru backup software yesterday (Friday), made a backup and cron made another during the night.

Client is leaving on a week long trip Monday morning, so he decided to restart the system to make sure it was fresh. It didn't restart. It failed in the boot - couldn't find boot file.

Client brought the machine over. He had a LoneTar crash recovery floppy set. It went through the motions, but didn't do a damned thing for us.

I got out that other hard disk, and installed SCO Unix on it. Restarted a few times to make sure it worked, Set up the tape drive, relinked the kernel, and restarted.

IO ERR - - then nothing - just like the last RAID controller that failed.

So a failing RAID controller had screwed the boot.

God help you if a RAID controller fails and you don't have an identical spare. Fortunately, the client knew this from the last episode, so he'd picked up a spare on eBAY.

The new controller booted the new drive fine, but the old one was still screwed.

So - I rebooted, installed bru, put in the tape and ordered bru to restore the whole system - - - UNREADABLE TAPE!
Tried the other tape - - - UNREADABLE TAPE!

Oh! Shit! Thought about this for a few minutes. Made a boot floppy set, attached the old hard disk, booted on floppy. The disk was still mountable, Backed up the root partition using the copy of bru we'd installed Friday night.

Now I'm going to have to figure out how to mount the /pos partition, and I haven't a clue how to do this in SCO (hint - it's not at all like Linux).

Reboot on the new disk, restore the tape, the root partition is pretty much intact. Reboot a couple of times.

Just for shits and giggles, I put one of the bad tapes in the drive and have bru try to read it. IT READS FINE! - restored the /pos filesystem.

Reboot. Need a couple of more files off that tape - - UNREADABLE TAPE!

SHIT! OK, I can get those files off the tape I made booted from the floppy . .

Reboot. Sitting staring blankly at the screen, an error message blinked and was gone - too fast to read, but I saw the word "tape" in it.

OH! SHIT! SHIT! SHIT! SHIT! SHIT!

Many long years ago, I had replace his 486 33-MHz Unix box with a faster 166-MHz unit, but he kept the old one as a backup. They had identical tape drives for easy transfer. Later, the tape drive in the new machine died, and we replaced it with a much higher capacity unit.

To make the new drive compatible with the old tapes and drive, I'd put a one line script, S82settape, in the rc2.d. It set the block size for SCSI tapes to 1024 - but it only worked if there was a tape in the drive during boot.

When we relinked the kernel for the DAT drive and rebooted, there was a tape in the drive, so our two backup tapes were written with the wrong block size. The reason I could read the Thursday tape that one time was I had restored the old startup files, then rebooted with a tape in the drive!

Things I did so long ago, just keep coming back to haunt me.

Now I'm going to go out and have a beer.

To quote Trudy, a business associate of mine who's a former mainframer, "I hate tape. There's just nothing worse than tape - except not having tape".

Removable hard disks have a number of problems, the greatest of which are bulk, fragility, cost and connector wear and contamination.

Looking at the system I was dealing with here, it's a complex, heavily used, 16-user Point of Sale / order tracking / production tracking system. It's prudent to rotate media daily on a 7 day cycle (plus a monthly cycle of Fridays if possible, and annual archives). and carry the most recent backup off-site.

Removable hard drives are too big to fit in purse or pocket and would be quickly destroyed by dropping and other accidents.

Yes, you can't trust a tape system. You have to auto-verify your backups daily and test restore every few weeks, but it's still the most practical, because it's fast, and the media is durable, portable, and cheap.

The death of tape has been predicted for every year in the past 15 years, but it's still with us, and will be for awhile.