Missing system tests after pubsub redesign

[tseaver]

The redesign effort (#3859) left behind only very minimal system tests. The older implementation had much broader system test coverage:

• Listing topics and subscriptions in the client's project.
• Creating subscriptions with non-default settings.
• Listing subscriptions bound to a topic.
• Setting / getting IAM policy for topics and subscriptions
• Creating / seeking snapshots.

I noticed the missing tests today when I went looking for examples of updating a topic's IAM policy.

[danoscarmike]

@tseaver could you please update the status of this? Have new tests been added? What is current coverage of the tests you enumerated here?

[theacodes]

The samples are likely a good starting place for system tests.

I mean, if anyone wants to take on moving the samples wholesale here I'd be cool with that.

[dhermes]

@danoscarmike No new tests have been added or are planned to be added.

As I identified bugs I made reproducible cases at https://github.com/dhermes/google-cloud-pubsub-performance and some of them may be worth turning into tests, but they run for too long to make that feasible.

What is current coverage of the tests you enumerated here?

Can you be more specific? Do you mean line coverage?

[danoscarmike]

Can you be more specific? Do you mean line coverage?

I meant which of the following points (per original issue post) have tests?

Listing topics and subscriptions in the client's project.
Creating subscriptions with non-default settings.
Listing subscriptions bound to a topic.
Setting / getting IAM policy for topics and subscriptions
Creating / seeking snapshot

If we are satisfied that behavior and performance are tested (one way or another) can we close this issue?

[tseaver]

I would say we should not close the issue until we have added those system tests: we can mark it as non-release-blocking, but the fact that we are missing tests which exercise what should be important use cases against the back-end should not be swept under the carpet.

[danoscarmike]

Fair. Thanks Tres.