Web Spider Crawler

Need a web spider to crawl pages and create an index of keywords and search terms based on targeting URLs and meta search. Site must return relevant and fast (under 1 second) results based on user query. Basically the web crawl/indexer would collect and contain an index of keywords, URLs and that the Admin designates through a web-based control panel that you create as part of this project. Indexed results would be diaplayed in web results pages that are displayed based on end user searches (similar to how any search engine works). Admin would type in keywords or URLs for websites and the spider would go crawl them for the index. End user can then tap this index by search box on the website. Need the crawler, Admin control panel to designate the crawl targets by keyword, web URL, search engines, etc. Need the page results generator script also. Basically it's a search site with Admin control on which sites to index and display in results. Only coders with examples of spiders in operation on the web today will be considered. The spider must perform direct crawl and be able to meta crawl through search engines that the Admin selects. Need complete source code (with comments), executables, etc. fully-installed and operational without bugs to run on the server. Basically this is a web crawler just like many available, with the ability to set the index to any URL or meta search keywords to crawl and build the index on. Must be efficient and low on RAM usage. User interface will resemble [login to view URL] ## Deliverables 1) Complete and fully-functional working program(s) in executable form as well as complete source code of all work done. 2) Deliverables must be in ready-to-run condition, as follows (depending on the nature of the deliverables): a) For web sites or other server-side deliverables intended to only ever exist in one place in the Buyer's environment--Deliverables must be installed by the Seller in ready-to-run condition in the Buyer's environment. The spider and Admin must be installed and fully-functional (bug free). 3) All deliverables will be considered "work made for hire" under U.S. Copyright law. Buyer will receive exclusive and complete copyrights to all work purchased. (No GPL, GNU, 3rd party components, etc. unless all copyright ramifications are explained AND AGREED TO by the buyer on the site per the coder's Seller Legal Agreement). ## Platform **ASP or ASP.NET SQL Web Windows server environment**