Hi,
Hi, I work as a Software Engineer in a company where my primary job is to crawl primary ecommerce websites in India. I am involved in designing a distributed crawler which can crawl websites like flipkart, snapdeal, paytm at a very high throughput. I have recently implemented one multithreaded crawler in python. Lets talk and let me understand what is that you need. The price also is approx because I do not know how much effort is needed.
Thanks,
Mayank