I need a web crawler for a video web site. The crawler will have two parts:
1) Crawl the whole site and record all historic data in a relational database
2) Crawl the site every X minutes/hours and record new added videos into the database
The site contains videos of famous TV series. What we need to fetch is for each TV series:
* Series name
* Category/Type
* IMDB rate
* Description
For each video in the series:
* Season/Number
* Video title
* Video descr
* Alternative video URLs.
Please respond if you have developed similar crawlers before. In your response, you need to explain the technologies you use. If you don't, i'll not consider your offer.
Merhaba,
Internet'ten veri toplama ve web otomasyonu konularinda uzmanim.
Insan ozellikleri tasiyan BOT gelistirebilim. JAVA programlama dili kullanirim ve yapacagim program windows uzerinde veya sunucunuz varsa orada calisabilir.
Gecmiste bunun gibi cok uygulama gelistirdim.
Iyi gunler