Build a php based crawler

We need to crawl in total 10 different sites Project plan // readme PHP based server-side application that.. Crawls through specified web content (multiple \'data URLs\', not just a single one.) (Data URL = data source web page) Data URLs can be looped (for loop for query string), extracted from an array, or crawled from a web page with certain rules to find the right URLs. Extracts specified data fields from the \'data URLs\', and forms a JSON object of them. Stores this JSON object into MongoDB using MongoDB PHP Driver. When executed, is able to send the crawled data from MongoDB in JSON to a remote server via HTTP POST request. Example of a page to be crawled (1): [login to view URL] Data URLs are found from this page, before the \'h3.job-header\' element. -> Example Data URL example: [login to view URL] Data to crawl from a data URL example: [login to view URL] Example of a page to be crawled (2): [login to view URL] Data URLs are found from this page as a first link element inside of \'tr.job-item\' table row. -> Example Data URL example: [login to view URL] Data to crawl from a data URL example: 1. [login to view URL] 2. [login to view URL] (Note that some of the data pieces will have to extracted from a text part, so certain amount of regular expressions // other data sorting methods are required here.) The application should be structured in a way, where core crawling & data processing models are separate from the data source specific crawler parts. By \'data source specific\', I mean that we\'ll want a separate files for each web resource to be crawled, for example \'Cvmarket\' for [login to view URL] specific rules / guides, and \'CVCrawler\' for [login to view URL] specific rules. These should extend the core crawling & data modelling parts of the application. Example: -> [login to view URL] executes multiple individual .php files, which each crawl a single data source. -> It simultaneously spits progress output (+ possible errors) in console + log file about the process. (example below) ### Storing the results to database.. ### ### Data stored successfully! ### Example application structure from our other crawler project: [login to view URL] (Here\'s zipped repo to examine the prevous project\'s code for guidance: [login to view URL]) 1. [login to view URL] <- cURL implement functions 2. [login to view URL] <- Bundless individual crawler files into one executable, and stores the results into MongoDB 3. [login to view URL], [login to view URL], [login to view URL] + the rest <- individual crawler files for 1 file per 1 data source. The MongoDB should look like this, when the [login to view URL] has been executed a couple times: [login to view URL]