[Issue-2741] use RetryingMetaStoreClient #2804
Conversation
| try { | ||
| try { | ||
| return CLIENT_CTOR.newInstance(hiveConf); | ||
| return RetryingMetaStoreClient.getProxy(hiveConf, null, HiveMetaStoreClient.class.getName()); |
There was a problem hiding this comment.
Maybe I'm mistaken, but this is the CTOR that is @VisibleForTesting?
Can we use:
public static IMetaStoreClient getProxy(
Configuration hiveConf, boolean allowEmbedded)
|
Actually on second look, it looks like ClientPoolImpl already does the retry logic (though no configurable retries), so if we replace it it will need to be a bigger replacement.. |
|
@jhsenjaliya do you plan to continue working on it, or I can also help? Probably we need to get rid of the extra retry at ClientPool layer and refactor more codes to get it to work. |
|
Hi @szehon-ho , yes, since the clientPool is designed to take care of reties, and RetryingHiveMetastore client does not really allow individual object creation, its hard to replace it without significant code replacement which i feel now unnecessary, I explained the issue in discussion at #2741 |
|
This has been implemented |
uses
RetryingMetaStoreClientto handle the connection failures automatically as per the hive internal logic. instead of trying to usehiveMetastoreClientdirectly.