Write a Simple C# Web Crawler Manager, using Windows Forms / WPF interface.
$10-30 USD
キャンセル
投稿日: 8年以上前
$10-30 USD
完了時にお支払い
Write a Simple Web Crawler Manager, using Windows Forms / WPF interface.
SCOPE:
- Should be possible to create a job, by specifying: the site name, starter urls for crawl (simple text area, one initial url per line), url patterns to download (one per line. if not empty, use CONTAINS to see if page should be download to disk), crawl deep, folder to download the html, and retry count.
- Should be possible to pause/resume any job. Pause should also occur if, internet connection breaks down, or the program is closed. When job is finished, should be possible to run it again. Should be possible to delete a job too.
- Should be possible to specify a proxy list. Turn on/off. If on, every time you make a request you get a different proxy from the list.
DETAILS:
- Must be fast.
- Must be Multithread.
- Must be written in C#.
- Could use any open source code, as frameworks like NCrawler or Abot to complete your job.
- Downloaded html files can have a pattern file name, like '[site name]-[date].htm'. You also can suggest me something.
- Application must have a job list, showing what is running, minor status.
- Application must have a simple textarea to show any relevant log/console message. Starting.../Downloading...Stopping.
TARGET SYSTEM:
Our system has .NET 4 and Microsoft SQLExpress.
DELIVERABLES:
We need a working sample with clean code including all source files in C#, that is able to index <[login to view URL]> with a link-depth of 3 and that can paused and resumed, when that pause and resume on disconnect internet connection and reconnect, or close program and open it again and resume the job.
Hi sir,
I am scraping expert, I have did too many similar projects, please check my feedback then you will know.
Can you tell me more details? then I will provide demo data for you.
Thanks,
Kimi
Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !Ready to start !
Hi,
I have good experience in developing crawlers, scrapers, large data mining and web automation projects for social media, classified and market place sites with multi-thread support.
I can develop this crawler as a stand-alone desktop application using C# for windows operating system. Please contact to discuss in detail about your requirements.
Thanks & Regards,
Charles
Hello, my name is Dmitriy. I'm interested in your job offer. One of the projects made on Freelancer was scraper for LinkedIn network, so I have required experience and pieces to boost development. If you interested in my offer please contact me so we could discuss details. I'm looking forward working with you soon.
Last Work Experience:
August 2012 – July 2014
Desktop - Web - Mobile Developer. Support - Setup - Deployment Specialist – ERP Developer at ERP DEVELOPMENT AS (Oslo, Norway), developing web applications with Microsoft .NET Framework 4 and mobile applications for iOS. Integrating .NET Framework 4 solutions with Microsoft Dynamics NAV. Microsoft Dynamics NAV Developments. Sharepoint Developments.
Reference: Hans Jorgen Lachmann. Phone: +47 90972310.