Error Connecting with Tinkerpop gremlin

[michaelawarren]

I'm trying to connect to Arcade DB with the gremlin tinkerpop api remotely. I'm not sure what I am missing. I've been able to connect to orientdb successfully using the tinker tinkerpop api and would like to be able to reuse the rest of my code i had for OrientDB.

I'm creating the traversal like this:

import org.apache.tinkerpop.gremlin.driver.Cluster;
import org.apache.tinkerpop.gremlin.driver.remote.DriverRemoteConnection;
import org.apache.tinkerpop.gremlin.process.traversal.AnonymousTraversalSource;
import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;

var cluster = Cluster.build()
            .addContactPoint("localhost")
            .port(8182) // also tried port 2480
            .credentials(user, password)
            .create();

var g = AnonymousTraversalSource.traversal().withRemote(DriverRemoteConnection.using(cluster));
// i want to run command like this
g.V().toList();

I'm starting the ArcadeDB up with docker as such

docker run -d --name arcadeDb `
    -p 2424:2424 -p 2480:2480 -p 8182:8182 `
    -e "arcadedb.server.rootPassword=passwordhere" `
    -e "arcadedb.server.defaultDatabases=Imported[root]{import:https://github.com/ArcadeData/arcadedb-datasets/raw/main/orientdb/OpenBeer.gz}" `
    arcadedata/arcadedb:latest

There error i am getting is:

java.lang.RuntimeException: Could not initialize client for Host{address=localhost/127.0.0.1:8182, hostUri=ws://localhost:8182/gremlin}
	at org.apache.tinkerpop.gremlin.driver.Client$ClusteredClient.lambda$new$4(Client.java:571) ~[gremlin-driver-3.4.11.jar:3.4.11]
	at org.apache.tinkerpop.gremlin.driver.Client$ClusteredClient.lambda$null$0(Client.java:529) ~[gremlin-driver-3.4.11.jar:3.4.11]
    ...
Caused by: java.util.concurrent.CompletionException: Could not initialize 2 (minPoolSize) connections in pool. Successful connections=0. Closing the connection pool.
	at org.apache.tinkerpop.gremlin.driver.ConnectionPool.<init>(ConnectionPool.java:128) ~[gremlin-driver-3.4.11.jar:3.4.11]
	at org.apache.tinkerpop.gremlin.driver.ConnectionPool.<init>(ConnectionPool.java:78) ~[gremlin-driver-3.4.11.jar:3.4.11]
	at org.apache.tinkerpop.gremlin.driver.Client$ClusteredClient.lambda$new$4(Client.java:565) ~[gremlin-driver-3.4.11.jar:3.4.11]
	... 5 common frames omitted
Caused by: org.apache.tinkerpop.gremlin.driver.exception.ConnectionException: Could not open Connection{host=Host{address=localhost/127.0.0.1:8182, hostUri=ws://localhost:8182/gremlin}}, {channel=eed24e1c}
	at org.apache.tinkerpop.gremlin.driver.Connection.<init>(Connection.java:149) ~[gremlin-driver-3.4.11.jar:3.4.11]
	at org.apache.tinkerpop.gremlin.driver.ConnectionPool.lambda$new$0(ConnectionPool.java:102) ~[gremlin-driver-3.4.11.jar:3.4.11]
    ...
	... 3 common frames omitted
Caused by: org.apache.tinkerpop.gremlin.driver.exception.ConnectionException: Could not complete connection setup to the server. Ensure that SSL is correctly configured at both the client and the server. Ensure that client WebSocket handshake protocol matches the server. Ensure that the server is still reachable.
	at org.apache.tinkerpop.gremlin.driver.Channelizer$WebSocketChannelizer.connected(Channelizer.java:268) ~[gremlin-driver-3.4.11.jar:3.4.11]
	at org.apache.tinkerpop.gremlin.driver.Connection.<init>(Connection.java:120) ~[gremlin-driver-3.4.11.jar:3.4.11]
	... 8 common frames omitted
Caused by: java.lang.RuntimeException: WebSocket channel=[eed24e1c] closed before the handshake could complete. Server logs could contain the reason for abrupt connection disconnect or the server might not be reachable from the client anymore.
	at org.apache.tinkerpop.gremlin.driver.handler.WebSocketClientHandler.channelInactive(WebSocketClientHandler.java:82) ~[gremlin-driver-3.4.11.jar:3.4.11]
    ... 
	... 1 common frames omitted

What am I missing? Is there a better way?

[lvca]

How did you run the Gremlin server with OrientDB?

[michaelawarren]

I ran it by using the defaults in their tinkerpop docker image

docker run -d --name orientdb `
    -p 2424:2424 -p 2480:2480 -p 8182:8182 `
    -e ORIENTDB_ROOT_PASSWORD=passwordhere `
    orientdb:3.2.2-tp3

[lvca]

@michaelawarren ah ok, you're using the specific tp3 distro. The good news is that we have a brand new GremlinServer plugin in the latest snapshot, you can start it with (be sure to remove the previous docker image, so a new pull of the latest is required):

docker run -d --name arcadeDb `
    -p 2424:2424 -p 2480:2480 -p 8182:8182 `
    -e "arcadedb.server.rootPassword=passwordhere" `
    -e arcadedb.server.plugins=GremlinServer:com.arcadedb.server.gremlin.GremlinServerPlugin `
    -e "arcadedb.server.defaultDatabases=Imported[root]{import:https://github.com/ArcadeData/arcadedb-datasets/raw/main/orientdb/OpenBeer.gz}" `
    arcadedata/arcadedb:latest

For more info: #207.

The updated docs are in https://docs.arcadedb.com/#Gremlin-API.

Bear in mind that you can create a gremlin-server.yaml under ArcadeDB's config directory and ArcadeDB will use it to configure your server. Or you can use the ArcadeDB configuration system by prefixing the setting with gremlin. and those properties will be passed to the Gremlin Server configuration.

Please let me know if works.

[michaelawarren]

I'm still getting the same error when trying to connect to port 8182. I do see that the gremlin-server has started up. Not sure what I am missing.

[michaelawarren]

I made progress. I added a gremlin-server.yaml with the host set to 0.0.0.0 i tried to set an env gremlin.host but that didn't seem to do anything. Still trying to make this work but it may be on the application side from here.

[michaelawarren]

At this point i think my issue is just figuring out how to configure the gremlin server. Also I realized it is arcadedb.gremlin.host still gives me different results but that is tinker pop's gremlin server. Thank you for your help. I hope i can figure out the correct config for the gremlin-server.

[lvca]

Try passing gremlin.host without the arcadedb prefix

[michaelawarren]

I've tried both with and without the arcadedb prefix and even with both

docker run -d --name arcadedb `
    -p 2424:2424 -p 2480:2480 -p 8182:8182 `
    -e "arcadedb.server.rootPassword=playwithdata" `
    -e gremlin.host=0.0.0.0 `
    -e arcadedb.gremlin.host=0.0.0.0 `
    -e arcadedb.server.plugins=GremlinServer:com.arcadedb.server.gremlin.GremlinServerPlugin `
    -e "arcadedb.server.defaultDatabases=Imported[root]{import:https://github.com/ArcadeData/arcadedb-datasets/raw/main/orientdb/OpenBeer.gz}" `
    arcadedata/arcadedb:latest

I did notice that the OrientDB tinkerpop exposes port 8182 and arcade db doesn't. https://github.com/orientechnologies/orientdb-docker/blob/f64bde2c10f5179d35dfd42a68ee1065efc789a0/release/3.2.x/3.2.2-tp3/Dockerfile. Probably a dumb question but could that be part of the problem?

Other things i've tried based on orientdb's gremlin-server.yaml:

docker command

docker run -d --name graphdb `
-p 2424:2424 -p 2480:2480 -p 8182:8182 `
    -v $PWD/config:/home/arcadedb/config `
    -e "arcadedb.server.rootPassword=rootpwd123" `
    -e arcadedb.server.plugins=GremlinServer:com.arcadedb.server.gremlin.GremlinServerPlugin `
    -e "arcadedb.server.defaultDatabases=Imported[root]{import:https://github.com/ArcadeData/arcadedb-datasets/raw/main/orientdb/OpenBeer.gz}" `
    arcadedata/arcadedb:21.11.1-SNAPSHOT

gremlin-server.yaml

host: 0.0.0.0
graphs: {
  graph : ../config/Imported.properties
}
scriptEngines: {
  gremlin-groovy: {
    plugins: { org.apache.tinkerpop.gremlin.server.jsr223.GremlinServerGremlinPlugin: {},
               org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {classImports: [java.lang.Math], methodImports: [java.lang.Math#*]},
               org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: [../config/Imported.groovy]}}}}
serializers:
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { ioRegistries: [org.apache.tinkerpop.gremlin.arcadedb.structure.io.ArcadeIoRegistry] }}             # application/vnd.gremlin-v3.0+gryo
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV3d0, config: { serializeResultToString: true }}
          # application/vnd.gremlin-v3.0+gryo-stringd
  - { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV2d0, config: { ioRegistries: [org.apache.tinkerpop.gremlin.arcadedb.structure.io.ArcadeIoRegistry] }}         # application/json

Imported.properties

gremlin.graph=org.apache.tinkerpop.gremlin.arcadedb.structure.ArcadeGraph
gremlin.arcadedb.directory=/home/arcadedb/databases/Imported

if i left in the

-e "arcadedb.server.defaultDatabases=Imported[root]{import:https://github.com/ArcadeData/arcadedb-datasets/raw/main/orientdb/OpenBeer.gz}" `

I would get an error about the database being locked

[michaelawarren]

If i remove the

-e "arcadedb.server.defaultDatabases=Imported[root]{import:https://github.com/ArcadeData/arcadedb-datasets/raw/main/orientdb/OpenBeer.gz}" `

then i get am able to kind of query the gremlin server for any new vertex i created. But no properties are sent back. Also the arcade studio is not able to query the database.

updated java code:

    private String getAllVertex(GraphTraversalSource g) {
        var vertices = g.V().limit(100).toList();
        var properties = vertices.stream()
            .map(v -> v.toString() + " - " + Joiner.on(",").join(v.properties()));
        return properties.collect(Collectors.joining("\n"));
    }

    public GraphTraversalSource traversal() {
        var g = AnonymousTraversalSource.traversal().withRemote(DriverRemoteConnection.using(createCluster()));
        return g;
    }

    public Cluster createCluster() {
        return Cluster.build()
            .enableSsl(false)
            .addContactPoint("localhost")
            .port(8182)
            .credentials("root", password)
            .serializer(createSerializer())
            .create();
    }

    private MessageSerializer<?> createSerializer() {
        var kryo = GryoMapper.build()
                    .addRegistry(ArcadeIoRegistry.instance());

        //noinspection deprecation
        return new GryoMessageSerializerV3d0(kryo);
    }

[lvca]

Thanks for all the info here. I'm writing a test case based on your code to be part of our automatic tests.

[lvca]

Also, I've opened the port 8192 in DockerFile (together with MongoDB, Redis, and Postgres ports). Please let me know if that works.

[lvca]

I've created a PR to fix this issue:

#211

[lvca]

Will be merged as soon as the regression tests are green from the CI

[lvca]

Merged!

[lvca]

The working configuration is the following:

gremlin-server.yaml:

host: 0.0.0.0
graphs:
  graph: ./target/config/gremlin-server.properties
scriptEngines:
  gremlin-groovy:
    plugins:
      org.apache.tinkerpop.gremlin.server.jsr223.GremlinServerGremlinPlugin: {}
      org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin:
        classImports:
          - java.lang.Math
        methodImports:
          - 'java.lang.Math#*'
      org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin:
        files:
          - ./target/config/gremlin-server.groovy
serializers:
  - className: org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1
    config:
      ioRegistries:
        - org.apache.tinkerpop.gremlin.arcadedb.structure.io.ArcadeIoRegistry

gremlin-server.properties:

gremlin.graph=org.apache.tinkerpop.gremlin.arcadedb.structure.ArcadeGraph
gremlin.arcadedb.directory=./target/databases0/graph

gremlin-server.groovy (used by Gremlin Server to register the graph instance):

import org.apache.tinkerpop.gremlin.server.util.LifeCycleHook

def globals = [:]

// defines a sample LifeCycleHook that prints some output to the Gremlin Server console.
// note that the name of the key in the "global" map is unimportant.
globals << [hook : [
        onStartUp: { ctx ->
            ctx.logger.info("Executed once at startup of Gremlin Server.")
        },
        onShutDown: { ctx ->
            ctx.logger.info("Executed once at shutdown of Gremlin Server.")
        }
] as LifeCycleHook]

// define the default TraversalSource to bind queries to - this one will be named "g".
globals << [graph : graph.traversal()]

Code:

  @Test
  public void getAllVertices() {
    final GraphTraversalSource g = traversal();
    var vertices = g.V().limit(3).toList();
    Assertions.assertEquals(3, vertices.size());
  }

  private Cluster createCluster() {
    final GraphBinaryMessageSerializerV1 serializer = new GraphBinaryMessageSerializerV1(
        new TypeSerializerRegistry.Builder().addRegistry(new ArcadeIoRegistry()));

    return Cluster.build().enableSsl(false).addContactPoint("localhost").port(8182).credentials("root", DEFAULT_PASSWORD_FOR_TESTS).serializer(serializer)
        .create();
  }

  private GraphTraversalSource traversal() {
    return AnonymousTraversalSource.traversal().withRemote(DriverRemoteConnection.using(createCluster(), "graph"));
  }

[michaelawarren]

Thank you. This work does allow for me to connect to a database with my java application with the tinkerpop api! I think this original question is probably resolved.

[michaelawarren]

when will the arcadedb-gremlin 21.11.1 be available as a non snapshot?

[lvca]

Glad it worked! 80% should be released tomorrow, 20% in 2 days.