issues using /bin/plugin to install

[tyh0]

Hi there,

I am currently trying to use both elasticsearch and couchbase db in a GKE kubernetes cluster. Part of this involves installing the couchbase elasticsearch plugin.

My k8s cluster currently has operational couchbase pods, and I have added elasticsearch as described at https://github.com/pires/kubernetes-elasticsearch-cluster.

The instructions I am using to install the plugin are described at http://developer.couchbase.com/documentation/server/4.0/connectors/elasticsearch-2.1/install-intro.html.

According to the above plugin instructions, I must install the plugin in each instance of ES. I enter the ES containers (sudo docker ps -a, get the ES container id, then sudo docker exec -ti sh) and then attempt to run the install instructions as described above.

When I first did this, I would get the following error:
Exception in thread "main" java.lang.IllegalArgumentException: Could not resolve placeholder 'HTTP_CORS_ALLOW_ORIGIN'.

I then set environment variables in each the yaml files for the ES deployments as follows:
name: HTTP_CORS_ENABLE
value: "true"
name: HTTP_CORS_ALLOW_ORIGIN
value: /https?://github.com/couchbaselabs/elasticsearch-transport-couchbase/releases/download/*/

I had hoped this would alleviate the issue, but then when I repeated the process I got the same error, except now the environment variable having an issue was NETWORK_HOST.

I set the env variable similarly, this time name: NETWORK_HOST and value: "local" as per the ES documentation.

Now I am being presented with the same error except now the variable having an issue is DISCOVERY_SERVICE. When I check the link on https://github.com/pires/docker-elasticsearch-kubernetes regarding DISCOVERY_SERVICE, it brings me to another plugin page. When I try to install this plugin by running plugin install io.fabric8/elasticsearch-cloud-kubernetes/2.3.4, I get the same errors as when I try to install the ES-CB plugin.

Is there something I'm inherently doing wrong here? If so, what should I be doing to successfully install plugins?

Thanks for any and all help you can provide :)

[tyh0]

This is one of the yaml files that I am currently using to try to get around this issue:
https://gist.github.com/tyh0/4fa3cde4693c717166db57e86ed8ae6f

[tyh0]

bump

[pires]

I have to try and reproduce, otherwise I'm clueless.

[tyh0]

Here's some information that I can give you for reproduction purposes:

Currently the google container engine cluster contains couchbase pods as described in https://github.com/couchbase/kubernetes.

It also contains an nginx pod that serves as the entrypoint for the cluster, as described at https://github.com/kubernetes/kubernetes/blob/master/examples/https-nginx/nginx-app.yaml.

I've added to the cluster 1 es-master pod, 1 es-client pod, and 1 es-data pod, as described in https://github.com/pires/kubernetes-elasticsearch-cluster. These pods are using elasticsearch v2.3.4 as described in their documentation.

I'm currently trying to run the couchbase elasticsearch plugin as described at https://github.com/couchbaselabs/elasticsearch-transport-couchbase. The version I'm running is version 2.2.3.4, as this is the one that's supposed to be compatible with ES version 2.3.4.

The documentation at https://github.com/couchbaselabs/elasticsearch-transport-couchbase says to first install the ES Plugin on each node using the command below:

/elasticsearch/bin # ./plugin install https://github.com/couchbaselabs/elasticsearch-transport-couchbase/releases/download/2.2.3.4/elasticsearch-transport-couchbase-2.2.3.4.zip

However when I ssh into the VM, then exec into the docker container to do this, I get the following error:

Exception in thread "main" java.lang.IllegalArgumentException: Could not resolve placeholder 'DISCOVERY_SERVICE'
at org.elasticsearch.common.property.PropertyPlaceholder.parseStringValue(PropertyPlaceholder.java:128)
at org.elasticsearch.common.property.PropertyPlaceholder.replacePlaceholders(PropertyPlaceholder.java:81)
at org.elasticsearch.common.settings.Settings$Builder.replacePropertyPlaceholders(Settings.java:1179)
at org.elasticsearch.node.internal.InternalSettingsPreparer.initializeSettings(InternalSettingsPreparer.java:131)
at org.elasticsearch.node.internal.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:100)
at org.elasticsearch.plugins.PluginManagerCliParser.main(PluginManagerCliParser.java:61)

If you have any other questions regarding how to reproduce this, please let me know. Thanks for any and all assistance you can provide!

[Branor]

This looks to be a problem with the container not having the expected configuration settings for ES, not specifically a problem with the transport plugin. Are you able to successfully install other plugins? Which configuration file (elasticsearch.yml) is it actually using?

[pires]

DISCOVERY_SERVICE is an environment variable populated in containers by Kubernetes itself. Obviously, said service needs to be created as per instructions in this repo. Did you do anything differently?

[tyh0]

Hi guys, thanks for your responses.

@Branor, the yml files I've been using for this cluster can be found here: https://github.com/pires/kubernetes-elasticsearch-cluster, and the ES container they use is the one from this repo.

@pires I didn't realize that the instructions on this repo were mandatory. They only list the environment variables that may be used in replication controllers. I've since gone and tried to install the plugin shown at https://github.com/fabric8io/elasticsearch-cloud-kubernetes#kubernetes-pod-discovery, but this had issues as well. Also looking at this specific repo, I didn't see instructions indicating that I had to create the discovery service.

I looked at the dockerfile that is used to create the ElasticSearch containers, and it runs the shell script that is supposed to instantiate the environment variables. Isn't this the step that's supposed to prevent these issues, as the image the containers are built on is supposed to have the environment variables set. I was under the impression that the environment variables could be changed in the replication controller if necessary, but that the image primarily took care of setting them up.

[pires]

@tyh0 you're right. This repo is just used to build the image used in https://github.com/pires/kubernetes-elasticsearch-cluster.

[tyh0]

So is it possibly the case that the image doesn't have the environment variables set correctly, or in such a way that the plugin is able to recognize them?

[tyh0]

@pires The link in your above comment is a bit outdated. The command to install the plugin ultimately hits this repo: https://github.com/couchbaselabs/elasticsearch-transport-couchbase

The versioning convention of the plugin is such that version <version#> of ES would correspond to version 2.<version#> of the plugin. Version 2.3.4 of ES therefore uses plugin version 2.2.3.4. This is the version that I was attempting to install, and it definitely exists as can be seen at https://github.com/couchbaselabs/elasticsearch-transport-couchbase/releases/tag/2.2.3.4.

[pires]

@tyh0 I deleted my previous comment after I re-read your instructions. I am able to reproduce this and I'm still trying to figure out why this would happen. However, it seems that elasticsearch user, which is used to run Elasticsearch, doesn't have access to certain environment variables.

[tyh0]

Oh awesome! Thanks :) I'm glad to know that I'm not being stupid or otherwise making a mistake in how I'm trying to do this. Do you have any idea of a timeframe for when this could be straightened away? I ask because I'm currently a co-op student, and my work term is ending at the end of the month. Ideally I really want to get this finished before I depart from the company.