Introduction

A robot, as it pertains to the software definition, is a program that does a specific job unassisted by human intervention. A spider is a specific type of robot, in that it “crawls” the Web for information. Although the distinctions tend to blur somewhat in common usage, a program whose task is more oriented toward “writing” to the Web (such as one that posts to classified and/or Free-For-All “FFA” ad sites) is probably more correctly called a robot, in contrast to a search-engine spider, for example, which has no effect on the content of the pages it visits for the purpose of information-gathering.

There’s hardly a browsing task you can find that couldn’t be made considerably easier and more efficient with an appropriate robot application. I’ve created HSPost, a FFA-posting robot with greater flexibility than many of the others I’ve seen. I’ve also spidered the NASDAQ site to collect over 8,000 ticker symbols for NYSE, AMEX, and NASDAQ stocks, which I then screened through Yahoo! for low, high, and historic price data. (It sure beats paging through both Investor’s Business Daily and the “S&P Stock Guide” with bifocals!)

Through the use of an appropriately-timed sleep() function call, I’ve had spiders notify me when a given stock rose above or fell below a given price or when I won a periodically-held online contest in which I either had to look for my name or pick numbers at random. I also use a spider periodically to collect information about new FFA sites and add them to the HSPost database, and these actually do a fair job of parsing HTML forms and extracting the expected field names and their default values. And I’ve had two versions of AutoBid, based on the changing setup of the eBay™ bidding scripts, to do last-minute (or -second) “sniping” of stamp auctions! (Both versions are undoubtedly obsolete by now, however.)

[NOTE: The sniper code contained in this book is an updated version
that still works, as of this writing.]

If you haven’t already thought of numerous tasks for which you wished a spider was available, by the time you’ve finished reading this book, I hope and expect that you’ll not only see dozens of possibilities, but you’ll be well-prepared to take them on as your own projects. Let’s do it!