A look at the report card...
I have been on the website of practically every single data producing government and federal agency to update historical datatables for CMD-MD. So I would like to assign some grades.
EIA: A-
The energy information agency is outstanding. They have dug deep to produce long-ish records of crude and petroleum product production. They have made my life a lot easier, by caring about the data and how they present it. Now if only we could get the EIA data onto FRED. I have used the EIA to update datatables found in the Survey of Current Business which published fairly detailed accounts of commodity production, consumption, exports and prices from 1923 through 1991.
USDA: A
I think the USDA does a fine job. They dug through the trade data and created a U.S. agricultural trade balance that begins in 1975- that's not incredibly spectacular, but if they hadn't done it I would have had to rely on the Census Bureau which would have stopped me dead in my tracks.
US Treasury Department: B+
I like that they have put all of the monthly statements of the public debt online from the 1870's. That proved incredibly useful to me. Also their partnership with the St louis Fed through FRASER proved indispensable for me - but i don't know how much they were involved with that. They also have quite a bit of foreign capital flow data in txt and csv format which is nice. They would receive an A+ if they put all the historical series of the monthly bulletin in xls format, and launched a government bond database that included every single issue since the founding of the nation.
St. Louis Fed: A+
No comment necessary, so easy to use, and many historical series from the nber. A wonderful resource.
Census Bureau: C-
The Census Bureau has been around forever and is technically in charge of the trade data. There are detailed records of commodity imports and exports by month dating back to at least 1913, yet the trade database product offered by the Census Bureau only goes back to 1989, is really slow, and forces you to grab data in chunks, If you want to grab the dollar volume of copper exports and imports you have to make 4 different calls, because you can only grab 1989-1995 and then 1996-2019, and only separately for exports and imports. What's even more of a headache is that it spits out the monthly numbers into separate sheets for each year.
They are are also lacking in other areas, for example only including data from 1992 to the present for manufacturing orders and inventories when, when earlier renditions go back to at least the 1930's.
Fannie, Freddie, and Ginnie Mae: B-
Although it's great they publish detailed pool level security data, it's the distribution of their respective monthly summaries which earns them the B-. They should have all the series from the monthly summaries in full history available in an spreadsheet. Instead you have to open up up each pdf and copy the values into excel yourself. All three agencies do this. I should be able to know how much volume Fannie Mae has been doing from it's inception, instead this data is buried deep in other archived sources.
HUD/FHA: C-
Although one can commend the random spreadsheets that HUD publishes on their website, the FHA has been around since the 1930's. There should be aggregate monthly statistics on the gross volume of new insured mortgages, and the characteristics of the insured mortgages as well as the purpose of each new insured mortgage. Also it's nearly impossible to reach anyone who might have this spreadsheet lying around. I have yet to reach anyone who could help in any capacity, I've called and emailed and keep getting redirected to their website.
BLS: B
The BLS is not exceptional. I used to be able to make calls from their API, now I can't seem to pull a single full time-series. What the heck happened??
Federal Reserve Board: C-
It's 2019 and muni and corporate bond and equity issuance are STILL not in the DDP. Also i'm giving them a C- for making zero effort to put the Federal Reserve Bulletin and Annual Statistical Digest data into .csv files. I mean hurry up already. The Bank of England has a spreadsheet with centuries of data, so does Sweden, Norway and Switzerland. Is it really that hard?
Also the federal reserve has been collecting and PUBLISHING exchange rate data since 1914 so why do the bilateral exchange rates not include this history?
USGS: B
I'll give the USGS a B because they do have a lot of data history available. The reason they get a B is because it is almost ALL annual, and the monthly data they do have only covers the past few years. I see great potential for the USGS. They should also coordinate with FRED.
EIA: A-
The energy information agency is outstanding. They have dug deep to produce long-ish records of crude and petroleum product production. They have made my life a lot easier, by caring about the data and how they present it. Now if only we could get the EIA data onto FRED. I have used the EIA to update datatables found in the Survey of Current Business which published fairly detailed accounts of commodity production, consumption, exports and prices from 1923 through 1991.
USDA: A
I think the USDA does a fine job. They dug through the trade data and created a U.S. agricultural trade balance that begins in 1975- that's not incredibly spectacular, but if they hadn't done it I would have had to rely on the Census Bureau which would have stopped me dead in my tracks.
US Treasury Department: B+
I like that they have put all of the monthly statements of the public debt online from the 1870's. That proved incredibly useful to me. Also their partnership with the St louis Fed through FRASER proved indispensable for me - but i don't know how much they were involved with that. They also have quite a bit of foreign capital flow data in txt and csv format which is nice. They would receive an A+ if they put all the historical series of the monthly bulletin in xls format, and launched a government bond database that included every single issue since the founding of the nation.
St. Louis Fed: A+
No comment necessary, so easy to use, and many historical series from the nber. A wonderful resource.
Census Bureau: C-
The Census Bureau has been around forever and is technically in charge of the trade data. There are detailed records of commodity imports and exports by month dating back to at least 1913, yet the trade database product offered by the Census Bureau only goes back to 1989, is really slow, and forces you to grab data in chunks, If you want to grab the dollar volume of copper exports and imports you have to make 4 different calls, because you can only grab 1989-1995 and then 1996-2019, and only separately for exports and imports. What's even more of a headache is that it spits out the monthly numbers into separate sheets for each year.
They are are also lacking in other areas, for example only including data from 1992 to the present for manufacturing orders and inventories when, when earlier renditions go back to at least the 1930's.
Fannie, Freddie, and Ginnie Mae: B-
Although it's great they publish detailed pool level security data, it's the distribution of their respective monthly summaries which earns them the B-. They should have all the series from the monthly summaries in full history available in an spreadsheet. Instead you have to open up up each pdf and copy the values into excel yourself. All three agencies do this. I should be able to know how much volume Fannie Mae has been doing from it's inception, instead this data is buried deep in other archived sources.
HUD/FHA: C-
Although one can commend the random spreadsheets that HUD publishes on their website, the FHA has been around since the 1930's. There should be aggregate monthly statistics on the gross volume of new insured mortgages, and the characteristics of the insured mortgages as well as the purpose of each new insured mortgage. Also it's nearly impossible to reach anyone who might have this spreadsheet lying around. I have yet to reach anyone who could help in any capacity, I've called and emailed and keep getting redirected to their website.
BLS: B
The BLS is not exceptional. I used to be able to make calls from their API, now I can't seem to pull a single full time-series. What the heck happened??
Federal Reserve Board: C-
It's 2019 and muni and corporate bond and equity issuance are STILL not in the DDP. Also i'm giving them a C- for making zero effort to put the Federal Reserve Bulletin and Annual Statistical Digest data into .csv files. I mean hurry up already. The Bank of England has a spreadsheet with centuries of data, so does Sweden, Norway and Switzerland. Is it really that hard?
Also the federal reserve has been collecting and PUBLISHING exchange rate data since 1914 so why do the bilateral exchange rates not include this history?
USGS: B
I'll give the USGS a B because they do have a lot of data history available. The reason they get a B is because it is almost ALL annual, and the monthly data they do have only covers the past few years. I see great potential for the USGS. They should also coordinate with FRED.