GCD 0.1.7 release incompatible with v1beta3 emulator

[tom-haines]

I have posted repeatable tests to demonstrate the regressions here:
github.com/tom-haines/gcd-dot-one-seven-regression

There are various errors when trying to communicate with the latest release of the gcd emulator (v1beta3) that prevents 0.1.7 from being used with v1beta3. The same basic "connect and push tests" all work under 0.1.6.

I've included examples of the regression under both docker and localhost; using DNS lookups and IPv4 addressing. There are also scheme and scheme-less variants of the URL used.

[eddavisson]

Looks like it's calling /v1beta3/projects/sample-ds instead of /datastore/v1beta3/projects/sample-ds.

@ajkannan What's the correct way to configure gcloud-java to talk to the local emulator?

[eddavisson]

By the way, thank you for providing code for the repro!

[ajkannan]

@eddavisson, this looks like a problem of conflicting constraints in the v1beta3 client. When specifying a localhost, scheme is not allowed. As a result, gcloud-java strips scheme from user input. However, the check in DatastoreFactory's validateUrl fails when the input starts with "127.0.0.1" (an "http://" is necessary for IP addresses). As a result, input using IP addresses rather than hostnames will not work. gcloud-java uses "localhost" instead of "127.0.0.1", so we didn't encounter this problem in our tests. A fix will have to come in the client for IP addresses to work. Depending on which constraint is relaxed in the client, gcloud-java may also have to push a fix.

@tom-haines Thanks for pointing this out and providing repro code! I would like to make a suggestion if you are using code similar to your repro code to set up your own tests. gcloud-java's LocalDatastoreHelper class makes it simpler to set up the tests against the local emulator. (Note: this helper was renamed and refactored right after 0.1.7; we recommend you upgrade to the latest gcloud-java version 0.2.1. If you'd like to stay with 0.1.7, the relevant class is LocalGcdHelper). gcloud-java's test helper will eliminate the need to download and start the gcd script yourself, and it also has an options() method that returns options so that you don't have to construct them yourself. (see the package-info for an example).

[eddavisson]

I think the issue might be that gcloud-java's options object accepts a single host option and then uses a heuristic to decide whether to set the "local host" option or the "project endpoint" on the underlying DatastoreOptions object. If the IP were recognized as a "local host", it would set the "local host" option, and then this library's DatastoreFactory takes care of adding the scheme (and /datastore in the path).
https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/gcloud-java-datastore/src/main/java/com/google/cloud/datastore/spi/DefaultDatastoreRpc.java#L42

The "local host" option is a little awkward, but it provides an additional hint to our client not to try to get credentials from the local environment (which can be slow).

A couple of options come to mind off the top of my head:

1. Split out a separate "local host" option in gcloud-java.
2. Always pass raw IP addresses to this library's "local host" option.

[ajkannan]

If the IP were recognized as a "local host", it would set the "local host" option, and then this library's DatastoreFactory takes care of adding the scheme (and /datastore in the path).

When a user sets the host to "127.0.0.1:8000", gcloud-java correctly determines that the user is trying to set the local host. For that reason, I don't think splitting up the local host option in gcloud-java or passing IP addresses to the local host option would solve this particular problem.

To clarify what I was saying before, it's not possible to supply raw IP addresses to the local host option in this library because validateUrl requires scheme (new URI("some IP address without scheme") will throw an exception) but this library's DatastoreOptions won't accept scheme for localhost.

[eddavisson]

Fixed in 1.0.0-beta.2.

[email2liyang]

when will v1.0.0 released?

[ajkannan]

Hi @email2liyang, are you referring to v1.0.0 for the gcloud-java client library or version 1.0.0-beta.2 for Cloud Datastore's client (the client in this repository)? If you're asking about gcloud-java, I suggest opening an issue on that repository (located here), and somebody should get back to you on that. If you meant the latter, Cloud Datastore's client (which gcloud-java currently depends on) is available on Maven now (see here).

I just made a pull request to update gcloud-java with this fix (see googleapis/google-cloud-java#1002). Once that gets merged in and gcloud-java releases v0.2.2, you will be able to use IP addresses corresponding to your local machine in DatastoreOptions. My understanding is that waiting for the next gcloud-java release shouldn't be a blocker, since you can use hostnames (or even better, use options() in LocalDatastoreHelper as described in my previous post), but correct me if that's not the case.

[tom-haines]

Thanks @ajkannan for the prompt fix (and note your PR has now been merged into gcloud-java).

Regarding whether this is a blocker re use of hostnames: I don't think you can use hostnames as suggested, and hence remains a blocker for using v0.1.7 with the current emulator.

In regression repo, have a look at the last test (copied here):

Even when using a hostname with the emulator (here hardwired hostname "docker-host" to docker-machine ip dev in /etc/hosts )

mvn clean test -Dgcp_ip=docker-host

Will FAIL.

But will PASS under 0.1.6

mvn -f pom-0.1.6.xml clean test -Dgcp_ip=docker-host

example config:

$ docker-machine ip dev
172.16.10.157
$ cat /etc/hosts
172.16.10.157 docker-host

( I've mentioned "docker-host" here, but equally you could just run gcd.sh on any machine of course )

[ajkannan]

Hi @tom-haines, this happens to be a distinct issue from the one above. I've opened an issue (googleapis/google-cloud-java#1014) to track fixing it. The issue here is that the IP address for the Docker container isn't recognized as a local or loopback socket, so gcloud-java considers it a "project endpoint" rather than a local host. This means two things:

1. You should use scheme in the host URL (in 0.1.7, scheme is not allowed for local host, but is required for project endpoints).
2. The version of the gcd script you're running (and the version that gcloud-java uses right now) uses a slightly different URL construction than the actual Google Cloud Datastore URL. It requires an extra "/datastore" (e.g. http://<hostname>:<port>/datastore). This is going away in the next version of the local emulator. gcloud-java plans to add a fix so that your Docker example (provided you add scheme to the host URL) should work. This fix depends on both this library's client being updated and the next version of gcloud-java being released.

If you need a quick workaround to upgrade gcloud-java and communicate with gcd.sh running in a container, use the URL http://<hostname or IP>:<port>/datastore instead of <hostname>:<port> when constructing DatastoreOptions. Keep in mind you'll have to change the URL again to http://<hostname or IP>:<port> when you update to the future release of gcloud-java that contains the fix for googleapis/google-cloud-java#1014.

[email2liyang]

@ajkannan , I find another incompatible error when upgrading from v0.1.6 to v0.2.2
if you execute method below in v0.1.6, it works without problem, when upgrade to v0.2.2,it throws exception"java.lang.IllegalArgumentException: Could not parse key". we have millions of record stored in other system for google key, this error is blocking us to upgrade to latest version of gcloud-java.

could you pls fix? or is there any way we can convert these type of String into a parseable key in latest version of gcloud-java?

com.google.cloud.datastore.Key Key
        googleKey =
        Key.fromUrlSafe(
            "partition_id+%7B%0A++dataset_id%3A+%22datastore-test-1123%22%0A%7D%0Apath_element+%7B%0A++kind%3A+%22ip_pending_msg%22%0A++id%3A+5629499534213120%0A%7D%0A");

anything incompatible from the implementation of https://github.com/GoogleCloudPlatform/gcloud-java/blob/master/gcloud-java-datastore/src/main/java/com/google/cloud/datastore/Key.java?