create tool / script to scrape similarweb estimated visits of 1,000s of websites a month. Avoiding the API costs
プロジェクト予算$250 - $750 USD
I currently use SimilarWebs chrome plugin to tell me manually how many visitors websites get each month. I need to scale and automate this process. I could use the similarweb API but they're expensive so I am looking at other options.
I need a tool or script to be able to enter 100s/1000s websites at a time and for it to fetch/find SimilarWebs data on them:
1- Estimated Visits - 100% needed
but if possible I would also like the following values from the chrome plug in
2- the past 6 months of estimated visits - this will enable me to average the data for a better estimate.
3 - The rank of the website within its county
4 - the sector its in, and the rank within its sector - something Similar Web does very well.
Need to be careful of high volume of traffic coming from one location, but I am sure this is something that can be done. We could either have a script that visits the [url removed, login to view] website and enters each site, then finds and copies the relevant data into a excel/csv file. Or do the same process from the website direct, using the similarweb plug in. Go to site, click plug in, see and copy data. Both would in theory work.
I would likely do this for 3,000 websites a month to start.
I am interested in hearing about your experihence and how you could build me a tool to do this.
- The New York Times
- Wall Street Journal
- Times Online