The client has made the following changes to the job.
Client prefers freelancers from:
You are still able to submit a proposal for this job.
The client prefers freelancers from
a different location.
I require a perl script that downloads every master.gz from ftp://ftp.sec.gov/edgar/full-index/ .
There is a master.gz per quarter. the data is 1993-present. there is a directory per year, and a directory for each quarter in the year.
The script will unpack the gz file. It contains a master.idx file.This is a | delimited file. The script should scrape it and load the data into a MySql db.
It should also download the files referenced in each line, and save it in the same location in a target directory. There should be a config file. This file will dictate which type of forms will be downloaded (e.g. 10-K)
The script should be able to initialize the db at the option of a cmd line arguments. It should also take a start date and and end date and only download the data for those dates from the ftp.
The script should be well commented and have the necessary error handling. It should also print the which master.idx file is being processed and which .txt file is being downloaded.
Sign in or Register to see more