Dedicated web crawler / spider

** Note - this is a resubmission, the previous coder couldn't deliver on time *** I am looking a VB.NET application that can extract information 'objects' from a page. Using regular expressions I want to be able to specify what objects should be extracted. If the crawler errors on sites (can't find objects), it should write out the error to an XML based log file. Further requirments: Very stable, should run as multithreaded, automated spidering engine. We have a current application (vb 6, using DAO) that does something similar, but that needs major work. We have a pretty good idea of what we want, but lack the hands-on experience with VB.Net. When this project runs well, will be looking for extensions to the functionality already described. (ASP.NET user interface, webservice interface, more detailed pattern matching, cookie support, authentication support, SQL Server 2000 integration, etc) I am looking for someone who has lots of experience with crawlers and/ or the httprequest object! More details can be found in the attached zipfile. ## Deliverables 1) Complete and fully-functional working program(s) in executable form as well as complete source code of all work done. 2) Installation package that will install the software (in ready-to-run condition) on the platform(s) specified in this bid request. 3) Complete ownership and distribution copyrights to all work purchased. ## Platform Windows 2000, Visual Studion .Net, no beta's ## Deadline information For the accepted bidder, addition documentation on the use of Regular Experssions in .Net is available! This bid is being resubmitted because the previous accepted bidder couldn't do the job.