Run integration tests on external pull requests

[2opremio]

Right now we don't run integration tests on external pull requests because there is no way to securely use the credentials to handle the VMs.

This leads to the increasingly frequent external PRs to break master. For instance #2175

[2opremio]

#2135 is a great example of a PR in which we need integration tests for better quality assurance.

[2opremio]

CC @marccarre

[alban]

Jenkins' GitHub plugin has a bot that asks on external PRs if it is "ok to test". When someone whitelisted by the config comments "ok to test" on the PR (after testing that circle.yaml is not modified to print the credentials), Jenkins runs the test.

Could CircleCI have something like that?

[2opremio]

Related: #2193 and #2270

[2opremio]

As a remark, it's possible to mimic the same setup in a fork (e.g. https://github.com/kinvolk/scope have done it), it requires setting up your own infrastructure, which is hard and and creates a lot of friction.

[bboreham]

Why do we have "Build forked pull requests" turned off at CircleCI when there is a separate setting "Pass secrets to builds from forked pull requests" that meets the requirement stated at the top?

EDIT: this seems to have been fixed since.

[dholbach]

@2opremio You said you had some ideas to fix this. Do you think you could summarise them here?