C# Web Automation

The goal for this project is to perform some fairly simple web automation. The trick is not the web automation part but the speed and optimization. The project will need to be a C# application that is multi-threaded and uses a Worker-Master model. It is basically a recursive website scrape were the worker will record some data from a web page and add it to the end of the queue. Each of those web pages are in the same format and added to the queue. At one point there is a simple form to fill out and and submit as well, some of the pages will have it some will not. Basically all are the same format. The application itself must login to the website before it can start to scrape. The interface GUI must support a list of accounts and a maximum number of urls per account to access before swapping to another account. A second list allows the user to enter any number of text lines. These are the lines posted on the form, if present. A random line. So the only GUI elements will be: a) A list of the user accounts b) A list of response texts c) A start button d) Number of threads e) Current rate or pages per hour average f) A button to add starting urls (feed the initial monster) g) an export button to dump the list of pending links to a text file. If you would like to be considered answer the following: 1) How will you determine the correct way to login the website? 2) What technique will you use to optimize for speed? ## Deliverables 1) Complete and fully-functional working program(s) in executable form as well as complete source code of all work done. 2) Deliverables must be in ready-to-run condition, as follows (depending on the nature of the deliverables): a) For web sites or other server-side deliverables intended to only ever exist in one place in the Buyer's environment--Deliverables must be installed by the Seller in ready-to-run condition in the Buyer's environment. b) For all others including desktop software or software the buyer intends to distribute: A software installation package that will install the software in ready-to-run condition on the platform(s) specified in this bid request. 3) All deliverables will be considered "work made for hire" under U.S. Copyright law. Buyer will receive exclusive and complete copyrights to all work purchased. (No GPL, GNU, 3rd party components, etc. unless all copyright ramifications are explained AND AGREED TO by the buyer on the site per the coder's Seller Legal Agreement). ## Platform C# / Windows XP