Upgrade existing php scraper (prices from eshops into MySQL)
I am looking for someone who will update existing PHP scraper that collects prices from eshops and inserts them into one MySQL table (history is created).
1, Currently 6 products from 6 eshops are scrapped (36 combinations)
2, Price is inserted in MySQL as number - standardized as number with 2 decimals (without any string - currency etc).
3, MySQL has the following columns: id (auto increment), product name (scrapped), price (scrapped), url, weight (fixed parameter associated with specific url/product), datetime (when scrapped).
4, PHP is run be CRON to collect prices regularly (currently every 15 minutes).
5, The table includes history - each time CRON is run, a new row for each new price is inserted.
6, Should the scrap fail, it continues to another product.
Now I want to update the scraper so that it collects app 150 prices:
11 products from 14 shops
- add 5 products from 6 existing shops (already scraped)
- add 11 products from 8 new shops (not yet scraped)
I also need to add one more fixed parameter that will be inserted into the table by the scraper for each product - its ID (that will be matched with specification table).
And I need to optimize the performance of the scraper - preferably scrap all the prices at the same time. Shall the whole scraping take for example 3 minutes so that the price of the first product is not obtained at 12:00 while the last at 12:03. Preferably receive the prices simultaneously - so that these are comparable as they are changing every few seconds.
Experience with similar job required.
Please, provide your offerings - delivery, price, experience.