FRED no longer working

[jhoodsmith]

FRED seems to have changed the URL structure for downloading CSV.

I'm using latest development version of pandas-datareader.

In using the example from the docs I get:

pandas_datareader._utils.RemoteDataError: Unable to read URL: http://research.stlouisfed.org/fred2/series/GDP/downloaddata/GDP.csv

[oOo0oOo]

I can also confirm this bug. Getting the same Error.

Found another possible URL after a quick look but not sure if it works in all cases:
https://fred.stlouisfed.org/graph/fredgraph.csv?id=GDP

[rsvp]

Just for reference, here's the full URL for the GDP time-series
with all the qualifiers, as of 2018-01-07:

https://fred.stlouisfed.org/graph/fredgraph.csv?chart_type=line&recession_bars=on&log_scales=&bgcolor=%23e1e9f0&graph_bgcolor=%23ffffff&fo=Open+Sans&ts=12&tts=12&txtcolor=%23444444&show_legend=yes&show_axis_titles=yes&drp=0&cosd=1947-01-01&coed=2017-07-01&height=450&stacking=&range=&mode=fred&id=GDP&transformation=lin&nd=1947-01-01&ost=-99999&oet=99999&lsv=&lev=&mma=0&fml=a&fgst=lin&fgsnd=2009-06-01&fq=Quarterly&fam=avg&vintage_date=&revision_date=&line_color=%234572a7&line_style=solid&lw=2&scale=left&mark_type=none&mw=2&width=968

At which point, a CSV file can be downloaded. The only relevant qualifier
to the URL, as @oOo0oOo points out, seems to be: id=GDP
to get raw data, i.e. simplification to:

https://fred.stlouisfed.org/graph/fredgraph.csv?id=GDP

although the frequency (fq), transformation (=lin for raw?),
and begin/end dates are worth noting.

Secure Transport Layer

Old URL used only http, but the new one defaults to https. Good FRED!

Pull request made

Cross-ref #436 -- PR https://github.com/pydata/pandas-datareader/pull/436/files

Thanks to @jhoodsmith, utilizes just the "id" qualifier.
But currently the build fails, and needs further attention:
https://travis-ci.org/pydata/pandas-datareader/builds/326012086

[NumberChiffre]

Getting the same error and updated the url using fredgraph, thank you!

[oOo0oOo]

Just a quick note: It seems the fred has reverted the URL; the original URL works again...

[jhoodsmith]

Good news that the original is working again.

It might be worth following point by @rsvp above, and make better use of qualifiers. I used 'id' only in #436, as this followed the original code. However, better use of 'freq' and/or begin/end dates will reduce impact on FRED servers.

[jhoodsmith]

On a separate issue, I'm getting the same errors as reported by the #436 Travis build when I run tests locally. There seems to general problems working with OECD and Google Options.

[rsvp]

Working status of old URL

So for the stlouisfed.org domain, how is the redirection
between the "research" and "fred" subdomains taking place?

Any official announcements from FRED as to their intentions?
Perhaps it was an internal experiment accidentally brought
into production. Their phone number in the U.S. is
+1 (314) 444-8444.

2018-01-09T18:32:21Z Verified that the "old" research subdomain URL is operational,
providing raw data as previously expected. The "new" fred subdomain is stand-alone,
with no redirection taking place.

[bashtage]

Fixed