Ok, plan B and what I posted here are pretty much the same thing. Except with this idea, I would write a spider to parse the xml into an sql index. The sql index of course the huge file that is compressed in a way that also allows extremely fast searching and retrival of data.

in plan B, I would only need to write a program to query post numbers, and your script would extract all the date from xml and present it to the parser I wrote. If it is locked down the way you say it is, maybe we have to do it like this, but on the other hand, if I can down load it through ftp*, its not that locked down is it?

Anyway, I have scripts to do most of this already. So this project would be nothing new for me. They would just need modification and the installation of the right tools. I understand your desire to limit the complexity of the server, we could do this with one new peice of software: SQL, again, any brand, I can make it work.

SQL is great I stand by it 100%. especially for searching or generating dynamic content. I won't give out specifics of my database, but I estimate that if I did this with HZ, pulling up a post by number would take ~.3 seconds, and doing a search for text in the message body and subject wouLd be about ~.9 seconds.

also it just hit me, if we resort to 'plan b' i would not necessarily need your spider trapper script. I could just cycle throught the message history, if would be an even simpler modification to the script I used way back. What scuttles this though is we are a lot bigger now than we were back then, its better to host the script on HZ than to use one that forces the entire HZ message history to traverse the the long cluttered and dangerous internet pipes that span the pacific ocean.

In the end I would prefer we do a search engine that is housed on the server itself, but of course it is your server, and thus your call. And after all, whatever works is good. . . . I'll just have bear the site of the google logo on the search page. . . along with the **'dishonor and discrace' of being seen as a site that could not provide its own search capability ;-)

Regards,
Justin Kelly

*btw: what up with ftp anyway? every time I do an 'ls' it locks up. So I haven;t been able to use it for a while. I've tested it on multiple machines. . .
**Tougue planted in cheek