Scrapy framework


this is a quick job to check/review my spider settings in [url removed, login to view] the spider was working 1 week ago but now it get like 150 items in 1st minute of running the stop finding new events and get cancelled automatically after a 10 minutes i will invite you as member in scrapinghub to check and fix it its sports data spider collect live sports data from [url removed, login to view] and fill in mysql database direct to the point as this is urgent I have few spiders running on [url removed, login to view] and scraping sports schedules and live scores data from [url removed, login to view] and video streams ulrs / video highlight embed codes from 3 websites [url removed, login to view] , [url removed, login to view] , [url removed, login to view] all the extracted data are pushed to MySQL db we need to do some changes to the streams/hightlights spiders to collect more info as well as make the data extraction faster our live scores data spider collect the live scores real time data from the source every 2 minutes we need to make it faster to collect it every 1 minute after we optimize the data extraction and storage in db we need to create few python scripts that pull some data from the db and update few sections in a subreddit on [url removed, login to view] thru their api wrapper praw i love to divide it into few tasks according to priority : [ 1- update the streams/highlights spiders (now it doesn't get all the streams from the source also doesn't assign them all to our events "matches" in the db) 2- create reddit comment posting script that get the collected streams from db and post them in a comment on the corresponding match post within the subreddit 3- update the sports data spiders including the live score spider to make it take less time

スキル: MySQL, PHP, パイソン, ウェブ記事のスクラップ

さらに表示: sql server backend data web based app, data entry based home, Scrapy data mining, using scrapy data mining, data entry based project, framework massive data entry, using yii framework generate data report, data entry based, scrapy framework, receive data linux based server, data mysql based dropdown selection ajax, ace framework market data, web updating data entry data conversion based, data spiders, extracting data html based tags perl, online data entry based company, ninjava framework, framework prado mysql, best jsf framework, sms gateway program net framework sql freelance program, php mysql framework templates, jsp used client server framework, zoho java framework, framework igoogle netvibes, online examination tool spring framework

プロジェクトID: #12160506



Hi there - My name is Khorshed. I've read your brief and can see that you’d like to debug your spiders of I can help you get this done. I would love to discuss with you further about taking this proje もっと

$22 USD / 時間

Hi, Greetings !! We are a team of 35 full time, dedicated developers having expertise on native iOS & Android app development, Core PHP, Joomla, Magento, Wordpress, Moodle, Expression Engine, Prestashop, OpenCart, もっと

$23 USD / 時間

Hi, We are expert in designing as well in development. We will provide you quality of work. We will provide you our best.We want long term relationship with you. Please choose us and provide a chance to work wi もっと

$16 USD / 時間

Hello, I'm a web developer with over 7 years experience. I would like to discuss further this project. There are all my programming skills: *Front-end: - HTML, HTML5, JSON, REST API - JAVASCRIPT (Ajax, もっと

$24 USD / 時間

"IT FLEX SOLUTIONS has focused exclusively on the Web Development/Designing projects with 8+ years of service. Our Web Development/Designing expertise has helped clients to create e-commerce, social networking, real-es もっと

$15 USD / 時間

Hello, I am an PHP Developer with more than 7 years of experience and shall get this done in about 10 days' time. Let me know if you have any doubts / queries about my expertise, will be more than happy to addr もっと

$23 USD / 時間

I have 297 reviews check I am Eng. Tarek have a Master degree of Information Technology I am a web programmer Professional in programming Mobile Phone using (ionic), PHP もっと

$15 USD / 時間

Hi! We are your perfect outsourcing partner. We are a team of experienced developers with more than 8 years of development of different projects. Our main developing language is PhP. We are proficient in working with もっと

$23 USD / 時間

Hello, my name is Olya. I represent Ukrainian IT Company «Webbook». We provide website design and web and app development services for organizations, public and government institutions, company or private web-pages. もっと

$25 USD / 時間
$23 USD / 時間

Dear Client, Greetings from Flowgica technologies, I have experience with these skills. We do have similar experience doing something similar to yours therefore I am looking forward to discuss and move ahead. please ch もっと

$15 USD / 時間

Hi, I am interested to work with you on this project, I reviewed the requests you listed and I would like to discuss further details with you. Since 2006 I am managing a Web Design Agency located in Romania, which cur もっと

$31 USD / 時間

Hi! Data scraping is my main field of work and I have done a lot of projects that include scrapy. I sure I can help you with optimization of speed issues ( only one thing it may require moove to virtual server ) Re もっと

$23 USD / 時間
$20 USD / 時間

Hello, I want to show you all relevant Demo and Designs which is similar to your project completed previously. To make sure about the requirement set and customizations, I want to discuss this project with you further もっと

$25 USD / 時間

A proposal has not yet been provided

$20 USD / 時間

Hi, I can do this for you. Please send a massage in the PMB for details.......Best Regards flashsaiful

$22 USD / 時間
$22 USD / 時間

Hello, Its a pleasure to let you know that I've Completed and Delivered similar project before. All I need to work upon customization part, if we can proceed towards more discussion. I have gone through your project もっと

$22 USD / 時間

I have been in the software field for 12 years. I have the expertise in understanding the structure of the websites from which data needs to be scraped, as I am also a web designer. This helps me in precisely locating もっと

$25 USD / 時間