キャンセルされた

Financial Data Engine - for ATLASFSD - open to bidding

This application will crawl several financial data websites every 6 hours:

1) [login to view URL]

2) [login to view URL]

3) [login to view URL]

Scraping program should be written In Python, probably Scrapy or similar.

Data to be scraped will be organised as:

A) Providers - various types of financial institutions that will have various characteristics including a list of products as well as various other fields describing the company

B) Products - there are several different categories of products. There are also lots of fields associated with each product, and there will be several products sold by each provider in each category.

A freelancer will to design and implement the MySQL Schema for all of the data above, including all of the various fields to match the scraped data as well as the relations between the tables. This will be based on the format and relations in the data from the three sources above, and will have input from me.

The raw data will be put through several programs:

1) Check the scraped data has worked correctly (specific format for each field)

2) If error, then the product (and the changed field) will be listed in an 'Errors' list in the GUI for the user to check the scraper and adjust inputs to fix

3) If the value has changed then the product (and changed field) will be listed in a 'Changes' list in the GUI for the user to track changes in data scraped

GUI with 4 x different summary tables, each table to have search, filter and sort functions, and 2 x summary pages (product and provider):

1) List of providers and some summary details

2) List of products and some summary details

3) List of changes to the data since the last scrape (not scraper errors)

4) List of potential scraper errors to be resolved (data error or scraped value much different format to the last scrape based on rules for each field)

GUI to have a number of functions that perform several database queries and return the results (if any) to one of the summary pages:

1) List of providers and a set of specified summary details

2) List of products and a set of specified summary details

3) List of changes to the data since the last scrape (not scraper errors)

4) List of potential unresolved errors in recent scrapes (not just changes in field value)

Engine to create a fourth set of data recording every provider and product:

- Three data sources to be triangulated based on specific criteria that I will specify

- Functionality to change source of data for each product or provider

スキル: HTML, MySQL, PHP, ソフトウェアアーキテクチャ, ウェブ記事のスクラップ

さらに表示: website design

採用者について:
( 0件のレビュー ) Sydney, Australia

プロジェクトID: #12839702

22人のフリーランサーが、このジョブに平均$2440で入札しています

esolzsales

Hello, We are one of the TOP providers in Freelancer, with more than 600 reviews and ratings. You might be seeing lesser reviews, as Freelancer splits reviews according to the category you select. We have read the もっと

$3333 AUD 40日以内
(348件のレビュー)
9.9
A2Design

Hi, Nice project you have there, let us help you with it! Our team is Russian-Canadian. We code in PHP. Check our recent projects here [login to view URL] Here’s a little video about our team too! ht もっと

$2268 AUD 22日以内
(114件のレビュー)
8.6
webbookstudio

Hello, my name is Olya. I represent Ukrainian IT Company «Webbook». We provide website design and web and app development services for organizations, public and government institutions, company or private web-pages. もっと

$2500 AUD 25日以内
(178件のレビュー)
8.3
r4rony

Hello Client Greetings..........!! We are web based Development Company. We have been nominated in Top 100 web Vendors 2013. We have experts available on various technologies Magento, Wordpress, Joomla, Drupal, もっと

$2371 AUD 30日以内
(185件のレビュー)
8.5
londonlance

We are a London (Shoreditch) based web development studio. Please get in touch to discuss further. Following are some of our recent projects; [login to view URL] A social media post scheduler and manager もっと

$3000 AUD 30日以内
(43件のレビュー)
7.6
$2500 AUD 30日以内
(12件のレビュー)
7.1
samitalways4u

Hello , I read your job descriptions very well and I am glad to know you that I can do it. Please talk to me then we can discuss more in detail about this project. Please Give me Chance to work with You. Thank You..

$2105 AUD 30日以内
(111件のレビュー)
7.5
evadevlopers

Hi There, Ready To Start now. I have 4+ years experience in PHP as well as Wordpress, Magento,OpenCart, Prestashop, OsCommerce….and also in responsive CSS3, bootstrap, html5 . etc You can check our work http://www もっと

$2500 AUD 2日以内
(224件のレビュー)
7.0
esalem

Hello, I am VB, VBA, C#, WPF, Java, Delphi, PHP, ASP.NET, CAD and GIS developer, I developed many desktop applications and custom tools, macros and Add-Ins. For reference see my profile. I am ready to discuss and もっと

$4789 AUD 40日以内
(26件のレビュー)
7.2
Shopify

Hello, I want to show you all relevant Demo and Designs which is similar to your project completed previously. To make sure about the requirement set and customizations, I want to discuss this project with you further もっと

$2577 AUD 30日以内
(30件のレビュー)
7.2
hitech10

Thank you for your interest in partnering with Hitaishin infotech for your website development project. With well over 100,000 firms offering website development services, we know how challenging it can be to find th もっと

$2368 AUD 30日以内
(21件のレビュー)
6.9
mmadi

Dear Client, Greetings from Flowgica technologies, I have experience with these skills. We do have similar experience doing something similar to yours therefore I am looking forward to discuss more about more details a もっと

$2100 AUD 36日以内
(43件のレビュー)
7.0
SevenStarInfo

Hello ! We want to discuss about your project as we have experience in it. What Differentiates us from the other freelancers : # Experience of more than 5 years in Unity 3D, Xcode, CoCoa 2d, Phonegap and major もっと

$3000 AUD 30日以内
(67件のレビュー)
6.9
Beautistar

Dear client, I am a professional mobile app & web server developer with 6+ rich experience. I would like to work on your project and deliver the project. I have developed many applications such as social app, dati もっと

$2500 AUD 30日以内
(22件のレビュー)
6.6
yogeshssanwal

I am Yogesh sanwal, 5+ years experienced in website and e-commerce development using PHP/MySQL and Internet Marketing. Take a look to my last work: [login to view URL] [login to view URL] http: もっと

$2500 AUD 30日以内
(218件のレビュー)
6.9
Verz1Lka

Hello! I'm web-scraping and web-automation expert and i think i can help you. I use python language and SCRAPY framework. My scripts works on windows, mac or linux, but linux is preferably. I can schedule scripts on s もっと

$1500 AUD 6日以内
(108件のレビュー)
6.6
techiestiffy

Hey, I will do it in Java. I have 4 years of experience in Java . I have recently scraped several websites for my clients. You can check below link . This is the latest scraping I have done. [login to view URL] もっと

$1500 AUD 10日以内
(71件のレビュー)
5.9
mehedi276

"Rate and duration will be fixed on discussion" Dear Hiring Manager, I have gone through your job posting and become very much interested to work with you. I am an expert in these fields. I have already completed sev もっと

$2319 AUD 30日以内
(59件のレビュー)
5.6
BEAS24X7

Hi, This is from BEAS Consultancy and Services Pvt Ltd ( [login to view URL] ) . We are an India based full service web agency. We have checked your project requirement and We are sure we would be able t もっと

$2400 AUD 24日以内
(12件のレビュー)
4.7
techashu91

Hello Hiring Concern, Thanks for the job posting, I have reviewed all the links and requirements shared by you in the job description. I have confident enough to handle this task. I am ready to start immediately. もっと

$2500 AUD 30日以内
(16件のレビュー)
4.8