Scrapy framework

  • ステータス 終了
  • 予算 $15 - $25 USD / hour
  • Total Bids 40


this is a quick job to check/review my spider settings in [url removed, login to view] the spider was working 1 week ago but now it get like 150 items in 1st minute of running the stop finding new events and get cancelled automatically after a 10 minutes i will invite you as member in scrapinghub to check and fix it its sports data spider collect live sports data from [url removed, login to view] and fill in mysql database direct to the point as this is urgent I have few spiders running on [url removed, login to view] and scraping sports schedules and live scores data from [url removed, login to view] and video streams ulrs / video highlight embed codes from 3 websites [url removed, login to view] , [url removed, login to view] , [url removed, login to view] all the extracted data are pushed to MySQL db we need to do some changes to the streams/hightlights spiders to collect more info as well as make the data extraction faster our live scores data spider collect the live scores real time data from the source every 2 minutes we need to make it faster to collect it every 1 minute after we optimize the data extraction and storage in db we need to create few python scripts that pull some data from the db and update few sections in a subreddit on [url removed, login to view] thru their api wrapper praw i love to divide it into few tasks according to priority : [ 1- update the streams/highlights spiders (now it doesn't get all the streams from the source also doesn't assign them all to our events "matches" in the db) 2- create reddit comment posting script that get the collected streams from db and post them in a comment on the corresponding match post within the subreddit 3- update the sports data spiders including the live score spider to make it take less time


Looking to make some money?

  • 予算と期限を設定してください
  • Outline your proposal
  • Get paid for your work


    • Forbes
    • The New York Times
    • Time
    • Wall Street Journal
    • Times Online