Read More »
I use Amazon's EC2 for my web crawling infrastructure.
Much of my web scraping library is open sourced to make sharing solutions with clients easier: http://code.google.com/p/webscraping
I can provide scraped data in whatever format is required: CSV, XML, MySQL, etc. (Most clients prefer CSV)
I speak native English,...
Mar 10, 2015|Data Analysis|$1,290|Working
Aug 7, 2014|Other IT & Programming|$164|Completed
Jul 23, 2014|Other IT & Programming|$500|Completed
Mar 29, 2014|Software Application|$200|Completed
This is my usual workflow for web scraping projects:
1) Discuss what data is needed:
- Website URL
- Output fields (business name, address, phone, email, etc)
- Output format (csv, xml, mysql, etc)
2) Agree on the budget.
3) I prepare a sample and make adjustments based on feedback.
4) Once downloading has finished I send the full output.
I can still make changes to the output format at this time - all downloaded webpages are cached so it is quick to prepare an update.
If the output file is large I will provide a download link from my server.
And if downloading takes a while I can provide regular updates of the data if required.
Read More »
The University of Melbourne
2005 - 2008
Research Assistant in various computer science fields: computer vision, information retrieval, augmented reality