The client has made the following changes to the job.
Client prefers freelancers from:
You are still able to submit a proposal for this job.
The client prefers freelancers from
a different location.
We invite bids/ proposals from high-caliber firms/ individuals to build a top-notch shopping crawler website which will extract data from a large number of websites across different geographical locations. The distributed web crawler should be robust, able to crawl dynamic websites, perform very deep crawl and have features such as politeness, scalable in design, optimal freshness etc.
The extracted data will be parsed, indexed and stored in a scalable database for a user to query via website. We are not decided on technical design and architecture so would welcome suggestions/ proposals.
If you have previous experience in building a large scale crawler along with aesthetically pleasing aggregator website with high-volume of products then that's a huge plus.Website need to be designed in such way as to maximize affiliate revenue. The design and code should be top class.
Typical skills required will be - strong coding skills (C, PHP, PERL, Python etc) , database design and archit...
Sign in or Register to see more