How to test gcloud applications

[txomon]

Hello,

I have seen there are some efforts to create a local environment for testing, such as the datastore emulator, and the pubsub emulator, but I haven't seen more or documentation on how to use them.

One of the main blocks for the development with gcloud AFAI see is that there is not an standard to setup a integration testing environment. Certainly the ability to transparently connect and authenticate to upstream services is nice, but in order to develop, in the teams I work in, we really need to have a local testing environment.

The library is easy to mock, but I would highly appreciate having an standard way for developers have their testing environment, and documentation on how to do it.

I understand this effort is under gcloud beta emulators, but I have failed to find a way to integrate them with this library.

[daspecster]

Hey @txomon!
Are datastore and pubsub the main services that you need to emulate for your testing environments or are there other services that you rely on as well?

[txomon]

@daspecster

The documentation I am looking now is for Datastore and Pubsub, but these are the services we are already using:

• Appengine with credentials/service account
• Datastore
• Pub sub
• Metadata service from compute
• Cloud Logging - logs are already being taken through stdin/stdout but we would love to have this implemented in the library. I will open a ticket for this one.
• Cloud SQL - Is already testable because it's Mysql

If ongoing reported issues are resolved, we will be migrating also to:

• Storage
• Bigquery

Storage and bigquery are still waiting for approval and to have a proper environment, but the others are already in use.

[dhermes]

• The only emulators that exist (as of now) are only for Datastore and Pub/Sub
• Usage of the emulators is included in our CONTRIBUTING doc. However, the code is written to support the emulators as-is. That is, if you use the docs for the emulators (which say to run gcloud beta emulators FOO start and then $(gcloud beta emulators FOO env-init)$), your code will just talk to the emulator (we pick up the env. vars. that get set).
• More emulators is outside the scope of this library and something the service owners would need to provide

[theacodes]

@txomon if you were developing using another cloud provider's services, how would you do system testing?

I ask this question because the emulators are currently pretty limited, so the best recommendation for testing is to either do integration testing (with mocks) or do system testing against a separate project.

[txomon]

@dhermes

1. Ok, I will be checking for new modules if created
2. It looks nice, if that info could be documented in the official docs that would be awesome (for visibility)
3. Totally understand that, I was just giving a list on what we are using.

---

@jonparrott

We basically run with rabbitmq instead of pubsub, dynamodb has an emulator as datastore does, and the rest of the databases are opensource (Postgres/Mysql). Redis and memcached are both opensource, so no problems there, and for metadata we used environment variables.

I agree however that Google Cloud Store may not make much sense, but for BigQuery it does definitely make sense to have something (a provided mock or something).

[majuscule]

Hi all,

I wanted to second @txomon's request/suggestion. I am looking for a compute backend mock and was disappointed to find that no support is provided. I was unaware of gcloud beta emulators before reading this issue, but even if compute was supported (which it is not), I am wary of requiring a separate process for testing (see #2 (comment)).

To answer your question @jonparrott, AWS cloud services have relatively comprehensive mock coverage provided by https://github.com/spulec/moto.

google.appengine.ext.testbed seems to be the closest thing provided by Google to this, but is again lacking in service coverage.

[theacodes]

Thanks for the feedback.

To answer your question @jonparrott, AWS cloud services have relatively comprehensive mock coverage provided by https://github.com/spulec/moto.

This is mocking the library, whereas we prefer to provide an emulator that will work across languages. Once this library reaches 1.0 it might be worthwhile to look into providing some sort of mocking for the library, but I'm unsure how useful that'll be in light of the emulators.

I am wary of requiring a separate process for testing (see #2 (comment)).

We are as well, but that applies to unit tests which explicitly don't need to hit the live services. Our system tests do indeed hit both lives services and the emulators (if available).

[txomon]

In my case the idea of having a mocking library is that I can run unittests in parallel, however I really prefer to have perfect emulators than mocking libraries indeed.

…

On Wed, 30 Nov 2016 18:09 Jon Wayne Parrott, ***@***.***> wrote: Thanks for the feedback. To answer your question @jonparrott <https://github.com/jonparrott>, AWS cloud services have relatively comprehensive mock coverage provided by https://github.com/spulec/moto. This is mocking the library, whereas we prefer to provide an emulator that will work across languages. Once this library reaches 1.0 it might be worthwhile to look into providing some sort of mocking for the library, but I'm unsure how useful that'll be in light of the emulators. I am wary of requiring a separate process for testing (see #2 <#2> (comment)). We are as well, but that applies to *unit tests* which explicitly don't need to hit the live services. Our *system tests* do indeed hit both lives services and the emulators (if available). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1822 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAN7mottwruRLtrJrAf2kv-c6RFdLxb1ks5rDbvlgaJpZM4InkV5> .

[dhermes]

@txomon We do have support in the library for Datastore, Pub / Sub and Bigtable emulators. You can run our system tests against the emulators via:

$ cd google-cloud-python/
$ tox -e bigtable-emulator
$ tox -e datastore-emulator
$ tox -e pubsub-emulator

See: https://github.com/GoogleCloudPlatform/google-cloud-python/blob/d775489da48d594ce5173a4078af4a0d3d2863f4/CONTRIBUTING.rst#system-test-emulators