Create Scraper for [login to view URL] Classifieds Using Headless Chrome with Puppeteer in Python 3 -- 2

完了済み 投稿 4年前 着払い
完了済み 着払い

We need a scraper for KSL classifieds ([login to view URL]).

The scraper needs to use Puppeteer and Headless Chrome.

The scraper needs to be written in Python 3.

All requests need to be routed through Crawlera ([login to view URL]) in order to avoid getting banned. We already have a subscription and API keys for this.

The scraper needs to accept the following search / filter parameters:

keyword

min price

max price

category

seller type

listing posted timeframe

The scraper also needs to be able to run without any search parameters.

We already have a Google Cloud Platform Virtual Machine (Google Compute Engine Virtual Machine) Debian instance. This scraper needs to run on it.

We already have a MySQL 2nd Gen 5.7 instance running on Google Cloud Platform. The scraper needs to write all results to this database.

The scraper needs to store the following attributes about each individual listing in our MySQL database:

title

location

datetime posted

price

category

description

number of page views

number of favorites

listing number

The scraper needs to run on a schedule (every 15 minutes, for example). This schedule needs to be modifiable if we want to change it later.

We need the code. We will run the spider to scrape all listings and run continuously on our Google Compute Engine instance.

The scraper needs to update existing listings in the database when listing properties on [login to view URL] change. Example: a cell phone is posted for $400. The scraper finds the cell phone and stores it in the database. Two days later, the seller changes the price to $350. The scraper sees the ad again, sees that a listing attribute (the price) has changed, and updates the row in the database.

Python Linux ウェブ記事のスクラップ ソフトウェアアーキテクチャ Google Cloud Platform

プロジェクトID: #20373155

プロジェクトについて

15個の提案 リモートプロジェクト アクティブ 4年前

アワード:

zekovicm

Hi there,I am Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this project ! I can start immediately and finish it within the もっと

$174 USD 3日以内
(142レビュー)
7.6

15人のフリーランサーが、平均$203 で、この仕事に入札しています。

Mickelson

Hi Nice to meet you. I have enough experience in python script. Below the libraries are I used in past project. selenium, pandas, matplotlib, lxml, beautifulsoup, scipy, and other useful libraries. I have written some もっと

$200 USD 7日以内
(118件のレビュー)
6.8
Programmer59

Hello Sir, We are expert in data scrapping We are team of professional developers with having more than 7 year's experience in this field also I read your project description and We are interested in it but still we n もっと

$250 USD 7日以内
(73件のレビュー)
6.4
brianconey

Hello How are you i have full time and I can start to work immediately Please contact me and do let us discuss about your project Thanks for your posting Xu Han

$140 USD 7日以内
(59件のレビュー)
6.3
smsaurabhv

‌Hi, I have gone through your requirement to scrape lots of websites. I am EXPERT in building scraping tools /scripts. Hence, I can SURELY work on your project. I am having 4 YEARS of EXPERIENCE in developing PHP-PYTHO もっと

$111 USD 3日以内
(97件のレビュー)
5.8
p4logics

Dear Sir, I am interested in your project. I'm expert in web scrapping and web automation using selenium and jsoup, data management, data mining. I assure, I will do my best to work with you on your project to present もっと

$200 USD 4日以内
(43件のレビュー)
5.9
albertpopov46

Hello! I am glad to place a bid on your project. I am good at Python and its scraping modules including selenium, scrappy, and so on. I have read your description and I can do your project perfectly. I can do your pro もっと

$140 USD 7日以内
(32件のレビュー)
5.4
BlackEagle86

Hi, How are you? I am very interest in your project. As I am a professional Python,Scraping Expert with great experience and skills, I am sure that I can help you I'm ready to start to work on your project right now. もっと

$200 USD 7日以内
(8件のレビュー)
4.2
razajen

I am a professional data scientist from Scotland, I am currently working with a financial firm scraping financial data from various sources. I have the relevant experience needed for this project. Lets discuss more whe もっと

$250 USD 7日以内
(7件のレビュー)
4.1
hsingla2709

Hello dear hiring manager With my good experience in core python, I can scrape any website with the help of Scrapy and Selenium. till now I have scraped more than 200 websites which includes Linkedin, Facebook, Twitter もっと

$200 USD 2日以内
(2件のレビュー)
2.6
NoumanAndUzair

Hey there, I'm a professional website developer on python frameworks(flask). I can do your web scrapping. For further details, please message me.

$230 USD 7日以内
(0件のレビュー)
0.0
theukwebexpert

Hello there! My name is Chris, I'm a UK based and native English speaking web developer and app expert with 10 years experience building and maintaining complex web apps and websites. I've built and maintained applica もっと

$250 USD 1日以内
(0件のレビュー)
0.0