Geek Speak — HabitableZone

Geek Speak » in reply to Imagine there is a webpage somewhere

Several ways

VB6 has a handful of Internet controls that handle sending and receving over the Net (HTTP and FTP, for example), based on the Windows WinINET library, that I've found more than a little awkward to use. You also have the WebBrowser control, which is just a wrapper around the IE HTML rendering engine. You could use it to load a Web page, then, using JavaScript, tunnel inside the page and access DOM elements directly. You'd find the elements holding the weather data and read their .innerHTML property to get the text; and you can access A elements to read the href attribute to find where they point and then follow.

To me the most elegant way to do it, though, is to use the XMLHTTPRequest object that comes with Microsoft's MSXML library. The thing's a "transceiver"--you give it a URL and goes out and fetches the page; and you can optionally submit form data to a server and get back results. The results are available in raw text, as an XML document if it's XML data, and a Stream object. You can set request headers and read response headers. Works in synchronous or asynchronous modes; you'll want to use asynchronous and operate a pool of XMLHTTPRequests in a heavy-duty spidering situation ("10,000 mouse clicks" sounds heavy-duty to me!). The thing's simple as hell to use, and you can send and receive with a couple of dozen lines of code. You can use regular expressions (Regex) to quickly parse a returned HTML page, locating your data and further links to follow....lather, rinse, repeat.

BTW, XMLHTTPRequest runs everywhere, including in the browser, and Mozilla/FireFox has an equivalent object. XMLHTTPRequest forms the basis of "AJAX", the trendy Asynchronous JavaScript And XML that's The Next Big Thing. I've been using it since 2003, before it had a cutesy name, so right now I'm ahead of the buzzword curve. XMLHTTPRequest's a workhorse, and I don't know how I lived without it.

Log in or register to post.

The whole thread (20 posts)