The client has made the following changes to the job.
Client prefers freelancers from:
You are still able to submit a proposal for this job.
The client prefers freelancers from
a different location.
Your job will be to develop a Python script which will be running as part of a datawarehouse (Debian6), which does the following:
- monitors several folders for incoming text files, those text files contain several smaller independent "blocks"
- check if the blocks are already stored in the DB and if the blocks are valid
- parse each block for various text fields and fill DB fields according to the content
- "mass" load those blocks (each as one row) into the database (psql copy or batch inserts)
- delete the incoming text file
the final DB size will be in the TBs, so you need to have experience with larger DBs and need to come up with a good DB layout also.
If you are interested in this job, please provide short description/references of projects you did with
- multithreaded python applications (and how you implemented multithreading)
- big postgresql databases and database modeling
I will then provide a more detailed technical specifications documen...
Sign in or Register to see more