https://github.com/nuvolabase/orientdb/pull/1406 #1430

henryzhao81 · 2013-04-08T19:13:27Z

Fix #1406 which failed in Unittest.
2 Fixes :

Remove set className to previous for loop. Otherwise if in request JSON string, if @Class is not first object. In following loop, all fields type which before @Class will always be null.
If value instanceof ODocument and type is LINK, then save object.

Passed unit test

henryzhao81 · 2013-04-11T01:02:19Z

One more check in :
Check in change for OSchedule (#1226)
In orientdb-server-config.xml :
Add a new handler


< handler class="com.orientechnologies.orient.server.schedule.OScheduleHandler" >

< parameters>

< parameter name="enabled" value="false" />

< parameter name="databaseName" value="databaseName" />

< parameter name="user" value="admin" />

< parameter name="pass" value="admin" />

< /parameters >

< /handler >

A class will be created, each record represent a job : name : job name rule : cron like rule (*/5 \* \* \* *) : http://www.sauronsoftware.it/projects/cron4j/manual.php#p14 - 2. Scheduling patterns arguments : function parameters status : job status function : link to the job function (OFunction) starttime : last start time start : the switch to start or stop job

Once change enabled to true in orientdb-server-config.xml, timer thread will be start, every minute, it will check rule for each job, if rule is qualified, then the job will be started.

Create Job : name, rule and function are mandatory fields. Once job is created, by default start is false, status is STOPPED. Job name should be unique.
Start Job : update start to true, the status will be automatically change to WAITTING, and wait for time thread to check rule to decide if it is ok to run in each minute.
Run Job : if rule is qulified to run. Then the status will automatically change to RUNNING, and execute function. After function exectued, change back to WAITTING.
Stop Job : change start to false, the job will be stopped.

OScheduleHandler.java :
Contains TimerThread, to check each job rule and start the job if rule qualified.

OScheduler.java :
Each job.

OSchedulerListenerImpl.java :
Methods to retrieve, load , add and remove job.

OSchedulerTrigger.java :
Check and change job status.

henryzhao81 · 2013-04-11T01:09:15Z

One question : I run ant clean test, unit test passed on my Linux machine (ubuntu), but failed on my macbook.
Following are the error log on my macbook, not sure what's going on.

[testng] index intstanceof class com.orientechnologies.orient.core.index.OIndexTxAwareOneValue

[testng] FAILED CONFIGURATION: @BeforeClass setUp
[testng] java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: 10.7.2.31; nested exception is:
[testng] java.net.ConnectException: Operation timed out]
[testng] at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:357)
[testng] at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:267)
[testng] at com.orientechnologies.orient.test.database.auto.AbstractIndexReuseTest.getProfilerInstance(AbstractIndexReuseTest.java:67)
[testng] at com.orientechnologies.orient.test.database.auto.AbstractIndexReuseTest.setUp(AbstractIndexReuseTest.java:33)
[testng] at com.orientechnologies.orient.test.database.auto.SQLIndexWithoutSchemaTest.setUp(SQLIndexWithoutSchemaTest.java:40)
[testng] Caused by: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: 10.7.2.31; nested exception is:
[testng] java.net.ConnectException: Operation timed out]
[testng] at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:118)
[testng] at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:203)
[testng] at javax.naming.InitialContext.lookup(InitialContext.java:411)
[testng] at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1924)
[testng] at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1891)
[testng] at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:274)
[testng] ... 26 more
[testng] Caused by: java.rmi.ConnectException: Connection refused to host: 10.7.2.31; nested exception is:
[testng] java.net.ConnectException: Operation timed out
[testng] at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
[testng] at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
[testng] at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
[testng] at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:340)
[testng] at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
[testng] at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:114)
[testng] ... 31 more
[testng] Caused by: java.net.ConnectException: Operation timed out
[testng] at java.net.PlainSocketImpl.socketConnect(Native Method)
[testng] at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[testng] at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[testng] at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[testng] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[testng] at java.net.Socket.connect(Socket.java:579)
[testng] at java.net.Socket.connect(Socket.java:528)
[testng] at java.net.Socket.(Socket.java:425)
[testng] at java.net.Socket.(Socket.java:208)
[testng] at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
[testng] at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:146)
[testng] at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
[testng] ... 36 more
[testng] ... Removed 22 stack frames
[testng] FAILED CONFIGURATION: @BeforeClass setUp

FAILED: insertAvoidingSubQuery
[testng] com.orientechnologies.orient.core.sql.OCommandSQLParsingException: Error on parsing command at position #0: Error on parsing command at position #11: Class TEST not found in database
[testng] Command: INSERT INTO test(text) VALUES ('(Hello World)')
[testng] -------------------^
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.createException(OChannelBinary.java:517)
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.handleStatus(OChannelBinary.java:470)
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:145)
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:59)
[testng] at com.orientechnologies.orient.client.remote.OStorageRemote.beginResponse(OStorageRemote.java:1877)
[testng] at com.orientechnologies.orient.client.remote.OStorageRemote.command(OStorageRemote.java:1039)
[testng] at com.orientechnologies.orient.client.remote.OStorageRemoteThread.command(OStorageRemoteThread.java:261)
[testng] at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:59)
[testng] at com.orientechnologies.orient.test.database.auto.SQLInsertTest.insertAvoidingSubQuery(SQLInsertTest.java:238)
[testng] ... Removed 26 stack frames
[testng] FAILED: insertCluster
[testng] com.orientechnologies.orient.core.exception.ODatabaseException: Cannot open database
[testng] at com.orientechnologies.orient.core.db.raw.ODatabaseRaw.open(ODatabaseRaw.java:117)
[testng] at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:49)
[testng] at com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.open(ODatabaseRecordAbstract.java:116)
[testng] at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:49)
[testng] at com.orientechnologies.orient.test.database.auto.SQLInsertTest.insertCluster(SQLInsertTest.java:263)
[testng] Caused by: java.lang.IllegalStateException: Database demo is already open
[testng] at com.orientechnologies.orient.core.db.raw.ODatabaseRaw.open(ODatabaseRaw.java:94)
[testng] ... 26 more
[testng] ... Removed 22 stack frames
[testng] FAILED: insertSubQuery
[testng] com.orientechnologies.orient.core.sql.OCommandSQLParsingException: Error on parsing command at position #0: Error on parsing command at position #11: Class TEST not found in database
[testng] Command: INSERT INTO test SET names = (select name from OUser)
[testng] -------------------^
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.createException(OChannelBinary.java:517)
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.handleStatus(OChannelBinary.java:470)
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:145)
[testng] at com.orientechnologies.orient.enterprise.channel.binary.OChannelBinaryAsynch.beginResponse(OChannelBinaryAsynch.java:59)
[testng] at com.orientechnologies.orient.client.remote.OStorageRemote.beginResponse(OStorageRemote.java:1877)
[testng] at com.orientechnologies.orient.client.remote.OStorageRemote.command(OStorageRemote.java:1039)
[testng] at com.orientechnologies.orient.client.remote.OStorageRemoteThread.command(OStorageRemoteThread.java:261)
[testng] at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:59)
[testng] at com.orientechnologies.orient.test.database.auto.SQLInsertTest.insertSubQuery(SQLInsertTest.java:250)
[testng] ... Removed 26 stack frames
[testng] FAILED: insertWithNoSpaces
[testng] com.orientechnologies.orient.core.exception.ODatabaseException: Cannot open database
[testng] at com.orientechnologies.orient.core.db.raw.ODatabaseRaw.open(ODatabaseRaw.java:117)
[testng] at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:49)
[testng] at com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.open(ODatabaseRecordAbstract.java:116)
[testng] at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:49)
[testng] at com.orientechnologies.orient.test.database.auto.SQLInsertTest.insertWithNoSpaces(SQLInsertTest.java:224)
[testng] Caused by: java.lang.IllegalStateException: Database demo is already open
[testng] at com.orientechnologies.orient.core.db.raw.ODatabaseRaw.open(ODatabaseRaw.java:94)
[testng] ... 26 more
[testng] ... Removed 22 stack frames

lvca · 2013-04-11T01:23:27Z

Hi,
did you include the scheduler sources to avoid the library? Does it license allow it?

henryzhao81 · 2013-04-11T01:31:47Z

Yes, scheduler sources are included, total 6 classes.
The license is LGPL so some kind of attribution notice will be required. I leave the file header.

package com.orientechnologies.orient.server.schedule
OSchedulingPattern.java
OAlwaysTrueValueMatcher.java
ODayOfMonthValueMatcher.java
OIntArrayValueMatcher.java
OInvalidPatternException.java
OValueMatcher.jhava

Please check it out.

lvca · 2013-04-11T01:38:06Z

Cool! This classes could be used by Automatic Backup too in place of Timer. Backups with a cron expression would be very useful...

henryzhao81 · 2013-04-12T16:59:37Z

Hi Lvca, how is my change looks like. I found a potential issue, in distributed mode, if two server running same scheduled job, may cause conflict issue because the status is keep changing, and scheduled jobs on each server may be different for example some node may need backup some not may not. It is not make sense to synchronize or replicate Oschedule class in each node. My suggestion is maybe we can set some class/cluster which are not synchronizable into configuration. I am not sure if it is doable so far. If not, my suggestion is create another field in distributed-db-config.json called excluded, and put the class/cluster name in it which do not want to synchronize. How about that.

Code change are : ODistributedStorage.java

 
For example :
public OStorageOperationResult updateRecord(final ORecordId iRecordId, final byte[] iContent,
      final ORecordVersion iVersion, final byte iRecordType, final int iMode, final ORecordCallback iCallback) {
        //TODO : check configuration here, if class/cluster name in excluded list, just update local
        if (ODistributedThreadLocal.INSTANCE.distributedExecution || isExcludedClass(iRecoredId))
        // ALREADY DISTRIBUTED
      return wrapped.updateRecord(iRecordId, iContent, iVersion, iRecordType, iMode, iCallback);
 ...
}

lvca · 2013-04-12T20:37:22Z

Travis can't pass the tests. Is it aligned to the master head?

henryzhao81 · 2013-04-12T20:43:43Z

I just did. How about now.

#1406

henryzhao81 added 4 commits April 8, 2013 12:08

https://github.com/nuvolabase/orientdb/pull/1406

ea726c6

Fix orientechnologies#1406

Merge branch 'master' of git://github.com/nuvolabase/orientdb

7939799

https://github.com/nuvolabase/orientdb/issues/1226

248da8e

fix orientechnologies#1226

a77fd09

Merge branch 'master' of git://github.com/nuvolabase/orientdb

2d39aae

lvca added a commit that referenced this pull request Apr 12, 2013

Merge pull request #1430 from henryzhao81/master

2400ef4

#1406

lvca merged commit 2400ef4 into orientechnologies:master Apr 12, 2013

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

https://github.com/nuvolabase/orientdb/pull/1406 #1430

https://github.com/nuvolabase/orientdb/pull/1406 #1430

henryzhao81 commented Apr 8, 2013

henryzhao81 commented Apr 11, 2013

henryzhao81 commented Apr 11, 2013

lvca commented Apr 11, 2013

henryzhao81 commented Apr 11, 2013

lvca commented Apr 11, 2013

henryzhao81 commented Apr 12, 2013

lvca commented Apr 12, 2013

henryzhao81 commented Apr 12, 2013

https://github.com/nuvolabase/orientdb/pull/1406 #1430

https://github.com/nuvolabase/orientdb/pull/1406 #1430

Conversation

henryzhao81 commented Apr 8, 2013

henryzhao81 commented Apr 11, 2013

henryzhao81 commented Apr 11, 2013

lvca commented Apr 11, 2013

henryzhao81 commented Apr 11, 2013

lvca commented Apr 11, 2013

henryzhao81 commented Apr 12, 2013

lvca commented Apr 12, 2013

henryzhao81 commented Apr 12, 2013