re: a plan B
If you are going through the trouble of making a script to draw in a spider, then if google doesn't follow the trap, I could write a spider specifically to fall into it.
That would be a very clean approach. You might remember me bitching and moaning about the lousy security in wwwboard back when we first experimented with BBS software. Well, I put my money where my mouth is and locked up the crown jewels--the XML--behind multiple layers of security. You'd have to jump through a lot of hoops to get a script on the server to access the XML vault.
Did you put the historical data into the xml format when the server last crashed? how far back can we really go?
Have no fear, every word, every punctuation mark, every profanity, every incoherent random string, that's ever been typed into the Zone is preserved for all time in immortal XML. I converted all data from the old format into the new format when I switched over.
If you FTP in, you'll find a new 'backups' folder, and a subfolder with all the XML as of last night's mirror run, nicely compressed into Zip files. That's how I came up with the stats that Nick requested. You should download those for safekeeping, and to give yourself a look at the format.