Deprecated API Endpoint (Invalid JSON data returned from network!)

[Sillson]

Hey @ReagentX

Heads up that the fine folks at PurpleAir deprecated the use of the purpleair.com/json endpoint and developed their own API. https://api.purpleair.com/

My job requires me to process a large amount of PurpleAir data quickly, and at a high frequency. I was going to spike out on a new package.

[ReagentX]

Last I checked, there was no way to get API keys yet.

[Sillson]

Are you planning on retooling this package to support authenticating via a user's token, plus utilizing the new endpoints/responses? It's a fairly seismic shift.

[ReagentX]

The new API does not provide ThingSpeak IDs, so I am not sure how much of this package can be reused.

I would like this package to continue to work, however, but I need the new API to work first before I can try to make it compatible.

[Sillson]

@ReagentX Yes the new API has ThingSpeak IDs in the individual sensor channel. Again though, it is a large shift and would require an entire overhaul. Unfortunately I would know, I'm currently overhauling my own PurpleAir processing streams.

[thomastu]

Chiming in on this issue - it looks like key registration occurs via email. I was able to get a key by emailing contact@purpleair.com

[ReagentX]

Is there a date when the old API shuts off?

[thomastu]

According to the google doc linked here: https://www.purpleair.com/sensorlist the timeline is "soon", but the document seems to have been created in 2017. It might be worth dropping an email to ask.

NOTICE: PurpleAir is migrating all users to our new API at https://api.purpleair.com. Please email contact@purpleair.com to get a key to use it and you should know that we will be preventing access to the current url’s at www.purpleair.com/data.json and www.purpleair.com/json soon

[dulitz]

The JSON endpoint is unreliable for me now -- it is returning data for my sensor that doesn't match what's shown on the website (or, as far as I can tell, reality).

Is someone working on this? I haven't been using ThingSpeak at all, just polling the JSON, and intend to continue not to use ThingSpeak, so I'm not the ideal candidate to patch this up.

6 months ago when I first looked at PurpleAir's JSON document I don't remember seeing any deprecation section, but they now have big red text at the top saying we gotta switch to the new API real soon now.

[Sillson]

@dulitz I've been using the new API for the past several months -- and have worked with the PurpleAir team to add fields necessary to perform most actions/analysis from the sensors (synoptic) level querying. So far I've found it to be performant and very easy to use.

I have my own internal toolkit for the my lab that I've developed, but it's fairly specific to my needs. The PurpleAir API should be easy to implement. I was going to spike on a new package, but with the new API it felt unnecessary. feel free to HMU if you have any questions on how to use

[ReagentX]

Keeping this issue open as I intend v2.0 to leverage the current API. That said, the unit tests still pass for me. I am not sure why @dulitz is receiving bad data from the network, but this library does not alter the data it receives.

The new API documentation still does not provide instructions on where one can acquire an api_key.

[ReagentX]

Just for clarification–the point of this library is to provide a Pythonic interface to interact with air quality data.

Indexing with JSON/dict data is annoying and error prone. KeyErrors are annoying to debug and often lack context. This was my effort to make air quality data available to beginner programmers. One of the ways the library accomplishes this is ensuring errors are handled gracefully and explicitly so the user knows what went wrong without firing up a debugger.

If the data that powers this tool is no longer available to the public, then effort to develop this library only serves some institutions, not independent developers, and those institutions should not rely on free labor to power their research.

[dulitz]

The data that powers this tool, i.e. the source data of the PurpleAir monitors that PurpleAir manufactured and we purchased, is (as far as I know, and I haven't researched it deeply) as available to the public as it was before. It was available under a TOU before and it is available under a TOU now. AFAICT the issuance of an api key is automatic; they didn't ask me why I wanted one when I emailed to request one, and they didn't at that point ask me to agree to anything.

Of course that could change at any time. I just don't observe that a change has happened thus far.

To be clear, I'm not using this library, so any inaccurate JSON data (specifically for temperature) has nothing to do with this code. I had been wondering whether I should just roll my own client and I see that the answer is yes.

[Sillson]

@ReagentX understood, and agreed that a library to help guide beginner users has value. As well as shortcuts to map/visualize/compare etc. One of the benefits of the API is that they introduced a sensor_index instead of having to rely on indexing JSON 👍. The value of the data may change if the API introduces a paywall -- but as I understand it, hosting an open JSON payload that has been abused was quite the engineering feat to keep up (and price).